System

Abstract

A system is provided.SOLUTION: This system is provided with a means for allowing a user to input past menu information, the kind of his or her desired dish, his or her desired calorie intake and his or her weak food, a means for receiving and storing the inputted information in a database, and a means for analyzing the stored information. A system comprising: AI means for generating a plurality of menu candidates in consideration of nutrition balance and calorie intake; means for transmitting the generated menu candidates; means for allowing a user to select a menu from the generated menu candidates; means for generating a necessary ingredient list based on the menu selected by the user; and means for providing the ingredient list and how to make each dish.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Planning daily meal plans and purchasing ingredients can be very stressful for those who prepare meals. This is further complicated by the need to consider health factors such as calories and vitamin intake. Conventional methods have not made it easy to provide balanced meals tailored to the needs of individual users. [Means for solving the problem]

[0005] To address these challenges, the present invention provides a system with the following features: a means for a user to input past menu information, desired types of dishes, desired calorie intake, and disliked foods; a means for receiving the input information and saving it in a database; an AI means for analyzing the saved information and generating multiple menu candidates taking nutritional balance and calorie intake into consideration; a means for transmitting the generated menu candidates to a user terminal; a means for the user to select from the generated menu candidates; a means for generating a list of ingredients based on the menu selected by the user; and a means for providing the ingredient list and instructions for each dish. This allows users to significantly reduce the time and effort required for menu planning and ingredient purchasing, enabling them to maintain a balanced and healthy diet.

[0006] "User" means an individual or entity that uses the system to enter, select, and review menu items.

[0007] "Menu information" refers to information about meals a user has eaten in the past and their future meal plans.

[0008] "Type of cuisine" refers to the category to which the cuisine belongs, such as Japanese cuisine, Chinese cuisine, or Western cuisine.

[0009] "Calorie intake" is the total number of calories you wish to consume in a day.

[0010] "Disliked foods" are types of foods that the user does not like or avoids due to allergies or the like.

[0011] "User Terminal" means a device (e.g., smartphone, tablet, computer) used by a User to access the System.

[0012] "Database" refers to a storage device within the system that stores user input information and generated menu information.

[0013] "AI" stands for artificial intelligence, an algorithm or program that analyzes user information and automatically generates appropriate menus.

[0014] "Menu suggestions" are suggestions for meal plans for the next month generated by AI based on the user's information.

[0015] The "ingredient list" is a list of ingredients needed to prepare the selected menu.

[0016] "How to make a dish" refers to the steps and methods for making each dish included in the selected menu. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The purpose of this system is to reduce the effort and stress involved in daily menu planning and ingredient purchasing for users, and to provide healthy and balanced meals. The following describes the embodiments of the present invention.

[0039] Entering user information

[0040] User device:

[0041] Users access a dedicated application using their smartphone, computer, or other device and log in to the system. They input menu information from the past month, the type of food they prefer (Japanese, Chinese, etc.), their desired calorie intake, and any foods they dislike (including allergens). For example, a user might input "toast and salad" for breakfast, "spaghetti" for lunch, and "gratin" for dinner in the past. They might also select "Japanese, Chinese" as their preferences, specify their calorie intake as "1700 kcal / day," and input "nuts, shrimp" as foods they dislike.

[0042] Receiving and storing data

[0043] server:

[0044] The server receives the information sent from the user's device and stores it in a database, thereby retaining the individual user information needed for subsequent processing and analysis.

[0045] Menu generation

[0046] server:

[0047] The server retrieves the stored user information from the database. It then analyzes this information using a machine learning algorithm to generate the ideal menu for the next month, taking into account nutritional balance and calorie intake. Specifically, the AI ​​analyzes the user's past eating history and health status, and automatically generates five menu candidates. For example, one of the generated candidates might be "Breakfast: Rice balls and miso soup, Lunch: Fried rice, Dinner: Grilled salted mackerel."

[0048] Presentation and selection of menu options

[0049] server:

[0050] The server sends the five generated menu options to the user's terminal.

[0051] User device:

[0052] The user reviews the menu options and selects the most suitable one. For example, the user selects "Option 1."

[0053] Generate a materials list

[0054] server:

[0055] The server determines a monthly menu based on the menu selected by the user. A list of ingredients is generated based on the determined menu. For example, an ingredient list based on the determined menu may include "rice, miso, mackerel, and vegetables."

[0056] Providing cooking instructions

[0057] server:

[0058] The server provides instructions for each dish included in the confirmed menu. Detailed cooking instructions are prepared in the form of text and video, and links to major external recipe websites are also displayed. For example, when it comes to how to make rice balls, text information such as "Step 1: Cook rice. Step 2: Shape the rice balls" is provided, along with a video explaining how to cook grilled salted mackerel.

[0059] User device:

[0060] Users can check the provided ingredient list and cooking instructions, then actually cook the dish, and refer to videos and links to improve the quality of their cooking.

[0061] In this way, users can efficiently plan and shop for ingredients, and easily prepare balanced, healthy meals.

[0062] The processing flow will be explained below.

[0063] Step 1:

[0064] User device:

[0065] The user logs in to the system. After logging in, a special input form is displayed. Using this form, the user enters menu information for the past month. Specifically, the user enters information such as "January 1st: Breakfast - toast and salad, lunch - spaghetti, dinner - gratin."

[0066] Step 2:

[0067] User device:

[0068] The user selects the type of cuisine they desire. For example, they select "Japanese, Chinese." They also input their desired daily calorie intake. For example, they input "1700 kcal / day." They also input foods they dislike or are allergic to. For example, they input "nuts, shrimp."

[0069] Step 3:

[0070] server:

[0071] It receives information entered from the user's device and stores it in a database, which allows the system to store information about the user's personal information and preferences for subsequent processing.

[0072] Step 4:

[0073] server:

[0074] The system obtains the user's menu information and preferences for the past month from the database. Based on the obtained information, it uses a machine learning algorithm to perform an analysis. Based on this analysis, it generates five menu candidates that take into account nutritional balance and calorie intake. For example, candidate 1 would be "Breakfast: rice balls and miso soup, lunch: fried rice, dinner: grilled salted mackerel."

[0075] Step 5:

[0076] server:

[0077] The five generated menu options are sent to the user's device.

[0078] Step 6:

[0079] User device:

[0080] The user reviews the five menu options displayed. After reviewing the details of each option, the user can select the option that best suits them. For example, the user can select "Option 1."

[0081] Step 7:

[0082] server:

[0083] Receives menu information selected by the user and determines a monthly menu based on the selection. Based on the determined menu, generates a list of required ingredients. For example, the list may include "rice, miso, mackerel, and vegetables."

[0084] Step 8:

[0085] server:

[0086] It prepares recipes for each dish based on the confirmed menu, including text-based cooking instructions and video-based cooking instructions, and also generates links to major external recipe sites.

[0087] Step 9:

[0088] User device:

[0089] Users are provided with a list of ingredients and instructions for each dish, and are guided through videos and links to create the dish, making it easy for users to prepare a balanced meal.

[0090] This series of steps allows the system to significantly reduce the effort required for users to create menus and purchase ingredients, and supports healthy eating habits.

[0091] Example 1

[0092] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0093] In today's busy lifestyles, planning daily meals and purchasing ingredients can be a time-consuming and stressful process. It is also difficult and time-consuming to create a plan that takes nutritional balance into account. As a result, achieving a healthy and balanced diet can be challenging.

[0094] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0095] In this invention, the server includes a means for allowing a user to input past meal plan information, favorite types of dishes, desired energy intake, and ingredients to avoid, a means for receiving the input information and storing it in a data structure, and a generative AI model means for analyzing the stored information and generating multiple meal plan candidates taking into account nutritional balance and energy intake, thereby enabling a user to efficiently create menus, purchase necessary ingredients, and easily prepare balanced, healthy meals.

[0096] "User" refers to a person who uses the system to create menus and purchase ingredients.

[0097] "Means of input" refers to the mechanism by which users use a terminal to provide the system with information such as past meal planning information, preferred types of food, desired energy intake, and ingredients to avoid.

[0098] "Data Structure" refers to the arrangement of information within the system for organizing and temporarily or permanently storing information received from users.

[0099] "Generative AI model" refers to an artificial intelligence algorithm that generates meal plan suggestions that take into account nutritional balance and energy intake based on stored user information.

[0100] "Display Device" means a device for visually presenting generated meal plan suggestions and other information to a user, including, for example, a smartphone screen or a computer monitor.

[0101] "Ingredients List" refers to the list of ingredients needed to prepare a meal based on the meal plan selected by the User.

[0102] "Cooking instructions" refers to content that provides instructions and methods for preparing a dish using ingredients in text and video format.

[0103] "Energy intake" refers to the amount of calories a user needs per day.

[0104] The system of this invention allows users to efficiently plan menus and purchase ingredients, and easily prepare balanced, healthy meals. This system is implemented primarily using a user terminal and a server.

[0105] Entering user information

[0106] User terminal

[0107] Users log in to a dedicated application using a device such as a smartphone or computer. After logging in, users are prompted to enter the following information:

[0108] Menu information for the past month

[0109] For example: "Breakfast: Toast and salad", "Lunch: Spaghetti", "Dinner: Gratin"

[0110] Desired type of cuisine (Japanese, Chinese, etc.)

[0111] Example: "Japanese food, Chinese food"

[0112] Desired calorie intake

[0113] Example: "1700 kcal / day"

[0114] Foods you dislike (including allergens)

[0115] For example: "Nuts, shrimp."

[0116] Receiving and storing data

[0117] server

[0118] The information entered by the user is sent from the user terminal, and the server receives this information, which is then stored in a data structure for further processing.

[0119] Menu generation

[0120] server

[0121] The server retrieves the stored user information from the data structure and analyzes it using the generative AI model. Specifically, it performs the following steps:

[0122] Obtain the user's dietary history and health information.

[0123] Data is input into a generative AI model to generate multiple meal plan options that take into account nutritional balance and energy intake.

[0124] Example: "Breakfast: rice balls and miso soup, lunch: fried rice, dinner: grilled salted mackerel"

[0125] Presentation and selection of menu options

[0126] server

[0127] The server sends the generated meal plan candidates to the user terminal.

[0128] User terminal

[0129] The user checks the displayed menu options and selects the most suitable one. The selected information is sent to the server and used for the next process.

[0130] Generate a materials list

[0131] server

[0132] The server determines the menu for one month based on the selected menu and generates a list of ingredients, which is then sent to the user's device.

[0133] Example: "Rice, miso, mackerel, vegetables"

[0134] Providing cooking instructions

[0135] server

[0136] Based on the confirmed menu, the server provides cooking instructions for each dish, including text and video formats, as well as links to major external recipe sites.

[0137] Examples: "How to make onigiri: Step 1: Cook the rice. Step 2: Shape the onigiri" or "Video explaining how to cook grilled salted mackerel"

[0138] User terminal

[0139] Users can check the provided ingredient list and cooking instructions, then actually cook the dish, and refer to videos and links to improve the quality of their cooking.

[0140] In this way, the system helps users efficiently plan, purchase, and prepare balanced, healthy meals.

[0141] "Please enter the user's menu information for the past month. Example: Breakfast: Toast and salad, Lunch: Spaghetti, Dinner: Gratin"

[0142] "Please select the type of cuisine you would like. Example: Japanese, Chinese"

[0143] "Enter your desired daily calorie intake. Example: 1700 kcal / day"

[0144] "Please enter any foods you dislike or are allergic to. Example: nuts, shrimp."

[0145] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0146] Step 1: Enter your user information

[0147] User terminal

[0148] Users log in to a dedicated application using their smartphone or computer, and then enter the following information when prompted:

[0149] Menu information for the past month (e.g., "Breakfast: Toast and Salad," "Lunch: Spaghetti," "Dinner: Gratin")

[0150] Desired type of cuisine (e.g., "Japanese, Chinese")

[0151] Desired calorie intake (e.g., "1700 kcal / day")

[0152] Foods you dislike (e.g., "nuts, shrimp")

[0153] The input data sent to the server includes this user information.

[0154] Step 2: Receiving and storing data

[0155] server

[0156] The server receives the information sent from the user device. Specifically, when the input data reaches the server's API endpoint, the server performs data validation and format conversion, then executes a query and stores it in the database in the appropriate format.

[0157] Input: User's diet history, favorite dishes, desired calorie intake, ingredients to avoid

[0158] Output: Saved user information is recorded in the database

[0159] Step 3: Create a menu

[0160] server

[0161] The server retrieves stored user information from the database. Based on the retrieved data, it uses a generative AI model to generate multiple meal plan candidates that take into account nutritional balance and calorie intake. Specifically, the server performs the following steps:

[0162] Query the database to retrieve the user's dietary history and related information

[0163] Input the acquired data into the generative AI model and start the analysis

[0164] As a result of the analysis, five types of meal plan candidates are generated.

[0165] Example: "Breakfast: rice balls and miso soup, lunch: fried rice, dinner: grilled salted mackerel"

[0166] Input: User information stored in the database

[0167] Output: 5 meal plan candidates generated

[0168] Step 4: Presentation and selection of menu options

[0169] server

[0170] The generated meal plan candidates are sent to the user's device.

[0171] User terminal

[0172] The user reviews the proposed meal plans and selects the most suitable one. Specifically, the application displays a list of meal plan candidates, and when the user taps to select one, the selection information is sent to the server.

[0173] Input: 5 meal plan options

[0174] Output: The meal plan selected by the user

[0175] Step 5: Generate a materials list

[0176] server

[0177] The server determines a monthly menu based on the menu selected by the user. It then generates a list of ingredients required for this determined menu. Specifically, the server analyzes the menu information and lists the ingredients required for each dish. The list is then saved in a database and sent to the user's device.

[0178] Example: "Rice, miso, mackerel, vegetables"

[0179] Input: Menu information selected by the user

[0180] Output: Generated materials list

[0181] Step 6: Provide cooking instructions

[0182] server

[0183] The server provides cooking instructions for each dish based on the confirmed menu. This is provided in both text and video format, and also displays links to major cooking recipe sites. Specifically, it retrieves the relevant cooking instructions from a recipe database, generates video links and text data, and sends them to the user's device.

[0184] Examples: "How to make onigiri: Step 1: Cook the rice. Step 2: Shape the onigiri" or "Video explaining how to cook grilled salted mackerel"

[0185] User terminal

[0186] Users can check the provided ingredient list and cooking instructions, then actually create the dish, especially by referring to videos and links to improve the quality of the dish.

[0187] Input: Confirmed menu information

[0188] Output: Cooking instructions in text and video format

[0189] In this way, the system helps users efficiently plan meals, purchase the necessary ingredients, and prepare balanced, healthy meals.

[0190] (Application example 1)

[0191] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0192] The objective of this invention is to provide healthy and balanced meals by reducing the effort and stress involved in daily menu planning and ingredient purchasing for users. It also aims to provide a system that allows users to easily select menus, efficiently obtain the necessary ingredients, and prepare meals. In particular, it aims to realize a system that, by linking with food delivery services, saves users the trouble of purchasing ingredients on the spot.

[0193] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0194] In this invention, the server includes: means for a user to input past menu information, desired types of dishes, desired calorie intake, and disliked foods; means for receiving the input information and saving it in a database; AI means using a generative AI model that analyzes the saved information and generates multiple menu candidates taking into account nutritional balance and calorie intake; means for sending the generated menu candidates to a user terminal; means for the user to select from the generated menu candidates; means for generating a list of ingredients based on the menu selected by the user and linking it to a food delivery company; and means for providing the ingredient list and instructions for each dish. This allows users to easily plan healthy meals and reliably obtain the necessary ingredients.

[0195] "User device" refers to an electronic device such as a computer, smartphone, or tablet that a user uses to input information.

[0196] A "database" is a collection of electronic files that systematically stores information received from users and manages it so that it can be easily accessed and searched later.

[0197] A "generative AI model" is a component of artificial intelligence that uses machine learning algorithms to analyze user input and generate optimal menus.

[0198] "AI means" refers to systems or programs that use artificial intelligence technology to analyze data and perform specific tasks.

[0199] "Meal candidates" refer to multiple meal plans generated based on user input.

[0200] "Ingredient Delivery Service Provider" refers to a service provider that delivers the necessary ingredients according to the ingredients list based on the menu selected by the user.

[0201] This invention aims to provide users with healthy, balanced meals by reducing the hassle and stress of daily meal planning and ingredient purchasing. The system consists of a smartphone application, a cloud server, and a generative AI model. Each function of the system and its implementation are described below.

[0202] 1. Enter your user information

[0203] User device:

[0204] Users access the dedicated application using their smartphone or tablet and enter their past meal plan information, desired type of food, desired calorie intake, and disliked foods. Specifically, users enter their meal history from the past month, their preferred food category (e.g., Japanese food, Chinese food), their daily calorie intake (e.g., 1700 kcal / day), and allergy information (e.g., nuts, shrimp).

[0205] 2. Receipt and storage of data

[0206] server:

[0207] The server receives the information sent from the user's device and stores it in a database, thereby obtaining individual user information that can be used for subsequent data analysis.

[0208] 3. Menu generation

[0209] server:

[0210] The server retrieves the saved user information and analyzes it using a generative AI model. The generative AI model, which implements a machine learning algorithm (using TensorFlow or PyTorch, for example), generates multiple menu candidates that take into account past eating history, preferences, nutritional balance, and calorie intake. Examples of generated menu candidates include "rice balls and miso soup" for breakfast, "fried rice" for lunch, and "grilled salted mackerel" for dinner.

[0211] 4. Presentation and selection of menu options

[0212] server:

[0213] The five generated menu options are sent to the user's device.

[0214] User device:

[0215] The user reviews the proposed menu options and selects the most suitable one.

[0216] 5. Material list generation and automatic delivery linkage

[0217] server:

[0218] Based on the menu selected by the user, a one-month menu is confirmed and an ingredient list is generated according to the confirmed menu. The generated ingredient list is linked to a food delivery system on the cloud, and the necessary ingredients are automatically delivered to the user. The food delivery company periodically delivers the necessary ingredients to the user based on the generated ingredient list.

[0219] 6. Providing cooking instructions

[0220] server:

[0221] The server provides detailed recipes for each dish in the generated menu, with cooking instructions in text and video formats and links to external recipe sites.

[0222] User device:

[0223] Users check the provided list of ingredients and cooking instructions, and then actually cook the dish. For example, detailed instructions for making onigiri are displayed, such as "Step 1: Cook rice. Step 2: Shape the onigiri." Users can refer to videos and links to improve the quality of their cooking.

[0224] Examples and prompts

[0225] Examples:

[0226] The user inputs "I'm allergic to nuts and shrimp, and I like Japanese and Chinese food," and sets their calorie intake. The next month's menu is generated, consisting of "rice balls, miso soup, fried rice, and grilled salted mackerel," and the necessary ingredients are delivered by a food delivery company.

[0227] Example prompt sentence:

[0228] The user enters information about the past month's menu, favorite types of food, calorie intake, allergy information, etc. This information is used to create a balanced menu for the next month. Favorite types of food are "Japanese" and "Chinese," calorie intake is "1700 kcal / day," and allergy information is "nuts and shrimp." The generated menu should be "Breakfast: rice balls and miso soup, lunch: fried rice, dinner: grilled salted mackerel."

[0229] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0230] Step 1:

[0231] The user's device inputs past menu information, desired type of food, desired calorie intake, and disliked foods. The user then launches the application and inputs their meal history for the past month, their preferred food category (e.g., Japanese, Chinese), their daily calorie intake (e.g., 1700 kcal / day), and allergy information (e.g., nuts, shrimp). This information is temporarily stored on the user's device and sent for later processing.

[0232] Step 2:

[0233] The user device sends the input information to the server, which receives it and stores it in a database. This information includes the user's dietary history, preferences, allergy information, calorie intake, etc. The database is structured to allow efficient access and searching.

[0234] Step 3:

[0235] The server retrieves the stored user information from the database and analyzes it using a generative AI model. The server passes the retrieved data as input to the generative AI model (using TensorFlow and PyTorch). The model analyzes the user's past eating history and preferences, and generates optimal meal suggestions taking into account nutritional balance and calorie intake. Five types of meal suggestions are generated as output.

[0236] Step 4:

[0237] The server sends the five generated menu candidates to the user's device. The user's device receives this information and displays it on the screen. The user reviews the presented menu candidates and selects the most suitable one. The information on the selected menu is temporarily stored on the user's device and then sent back to the server.

[0238] Step 5:

[0239] The server determines a monthly menu based on the menu selected by the user. It then generates a list of ingredients required based on the determined menu. The ingredient list lists the ingredients required for each dish in the selected menu. The generated ingredient list is then sent to a food delivery system on the cloud.

[0240] Step 6:

[0241] The food delivery company prepares the necessary ingredients based on the ingredient list received from the server and creates a delivery schedule. The delivery company delivers the necessary ingredients periodically based on the user's address information. This ensures that the user can obtain the ingredients they need.

[0242] Step 7:

[0243] The server provides detailed instructions for each dish in the generated menu. The cooking instructions are prepared in text and video formats and displayed within the application. Links to external recipe websites are also displayed. Users can check this information and use it as a reference when actually cooking.

[0244] These are the specific processing steps of the system program that realizes the application example. At each step, it is clear what information is required as input and what data processing and calculations are performed to obtain the output.

[0245] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0246] This system aims to reduce the effort and stress involved in daily menu planning and ingredient purchasing for users, and to provide healthy, balanced meals. Furthermore, by combining it with an emotion engine, it can also provide personalized menu suggestions based on the user's emotions. Below, we will explain an embodiment of the present invention when combined with an emotion engine.

[0247] Entering user information

[0248] User device:

[0249] Users access a dedicated application using their own smartphone, computer, or other device and log in to the system. They enter menu information from the past month, the type of food they want (Japanese, Chinese, etc.), their desired calorie intake, and any foods they dislike (including allergens). For example, a user might enter "toast and salad" for breakfast, "spaghetti" for lunch, and "gratin" for dinner in the past. They might also select "Japanese, Chinese" as their preferences, specify their calorie intake as "1700 kcal / day," and enter "nuts, shrimp" as their disliked foods.

[0250] Receiving and storing data

[0251] server:

[0252] The server receives the information entered from the user's terminal and stores it in a database, which stores the user's personal information and preferences in the system for subsequent processing.

[0253] emotion recognition

[0254] User device:

[0255] Users input their feedback on the food, their daily physical condition, and their mood. The emotion engine also recognizes emotions from the user's facial expressions and tone of voice. This information is sent from the user's device to the server.

[0256] Menu generation

[0257] server:

[0258] The server retrieves the user's stored information from the database, as well as the user's emotional data recognized by the emotion engine. Based on this information, the system performs analysis using a machine learning algorithm. Based on this analysis, it generates five menu suggestions that take into account nutritional balance, calorie intake, and the user's emotions. For example, if the user is feeling stressed, it will suggest a menu using ingredients that have a relaxing effect.

[0259] Presentation and selection of menu options

[0260] server:

[0261] The server sends the five generated menu options to the user's terminal.

[0262] User device:

[0263] The user reviews the menu options and selects the most suitable one. For example, the user selects "Option 1."

[0264] Generate a materials list

[0265] server:

[0266] Receives menu information selected by the user and determines a monthly menu based on the selection. Generates a list of ingredients required based on the determined menu. For example, the list may include "rice, miso, mackerel, and vegetables."

[0267] Providing cooking instructions

[0268] server:

[0269] The server provides cooking instructions for each dish included in the confirmed menu. This includes cooking instructions in text format and cooking videos. It also displays links to major external cooking recipe sites. For example, the server provides text information such as "Step 1: Cook rice. Step 2: Shape the rice balls" for how to make rice balls, along with a video explaining how to cook grilled salted mackerel.

[0270] User device:

[0271] Users can check the provided ingredient list and cooking instructions, then actually cook the dish, and refer to videos and links to improve the quality of their cooking.

[0272] Accumulating emotional data and reflecting it in the next proposal

[0273] server:

[0274] The system accumulates user emotional data and reflects it in the next menu suggestion. Based on the user's past emotional data, it is possible to suggest more optimized menus. By continuously learning from this data, the system's accuracy improves.

[0275] In this way, the system can utilize the user's personal information and emotional data to efficiently plan meals and purchase ingredients, making it easy to prepare balanced, healthy meals.

[0276] The processing flow will be explained below.

[0277] Step 1:

[0278] User device:

[0279] The user logs in to the system. After logging in, a special input form is displayed. Using this form, the user enters menu information for the past month. Specifically, the user enters information such as "January 1st: Breakfast - toast and salad, lunch - spaghetti, dinner - gratin."

[0280] Step 2:

[0281] User device:

[0282] The user selects the type of cuisine they desire. For example, they select "Japanese, Chinese." They also input their desired daily calorie intake. For example, they input "1700 kcal / day." They also input foods they dislike or are allergic to. For example, they input "nuts, shrimp."

[0283] Step 3:

[0284] server:

[0285] It receives information entered from the user's device and stores it in a database, which allows the system to store information about the user's personal information and preferences for subsequent processing.

[0286] Step 4:

[0287] User device:

[0288] Users input their feedback, daily physical condition, and mood. For example, they can enter comments such as "I feel stressed today" or "I feel relaxed today." The emotion engine then analyzes the user's facial expressions and tone of voice to recognize their emotions. This information is sent from the user's device to the server.

[0289] Step 5:

[0290] server:

[0291] The system receives emotional data sent from the user's device and stores it in a database, allowing the user's emotional state to be accumulated and used for future suggestions.

[0292] Step 6:

[0293] server:

[0294] The system retrieves user information and emotional data stored in the database. Based on this information, it uses a machine learning algorithm to perform analysis. Based on this analysis, it generates five menu candidates that take into account nutritional balance, calorie intake, and the user's emotions. For example, if the user is feeling stressed, it will suggest a menu using ingredients that have a relaxing effect.

[0295] Step 7:

[0296] server:

[0297] The five generated menu options are sent to the user's device.

[0298] Step 8:

[0299] User device:

[0300] The user reviews the five menu options displayed. After reviewing the details of each option, the user can select the option that best suits them. For example, the user can select "Option 1."

[0301] Step 9:

[0302] server:

[0303] Receives menu information selected by the user and determines a monthly menu based on the selection. Generates a list of ingredients required based on the determined menu. For example, the list may include "rice, miso, mackerel, and vegetables."

[0304] Step 10:

[0305] server:

[0306] It prepares recipes for each dish based on the confirmed menu, including text-based cooking instructions and video-based cooking instructions, and also generates links to major external recipe sites.

[0307] Step 11:

[0308] User device:

[0309] Users are provided with a list of ingredients and instructions for each dish, and are guided through videos and links to create the dish, making it easy for users to prepare a balanced meal.

[0310] Step 12:

[0311] server:

[0312] The system accumulates user emotional data and reflects it in the next menu suggestion. Based on the user's past emotional data, it is possible to suggest more optimized menus. By continuously learning from this data, the system's accuracy improves.

[0313] Example 2

[0314] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0315] Conventional menu creation systems only consider the user's preferences and nutritional balance, and are unable to provide personalized suggestions that reflect the user's mood or physical condition at any given time. As a result, users spend a lot of time and effort trying to find meals that suit their mood and physical condition, which can cause mental stress. This issue is especially important in today's world, where healthy meal choices that take into account nutritional balance and calorie intake are required.

[0316] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0317] In this invention, the server includes: means for a user to input past menu information, desired types of dishes, desired calorie intake, and disliked foods; means for receiving the input information and saving it in a database; means for analyzing the user's facial expressions and voice tone using an emotion recognition engine to generate emotion data; generation AI model means equipped with a machine learning algorithm that analyzes the saved information and emotion data and generates multiple menu candidates taking into account nutritional balance, calorie intake, and the user's emotions; means for transmitting the generated menu candidates to a user terminal; means for the user to select from the generated menu candidates; means for generating an ingredient list based on the menu selected by the user; and means for providing the ingredient list and cooking instructions for each dish. This enables personalized menu suggestions that reflect the user's emotions and physical condition, allowing users to easily prepare healthy and balanced meals.

[0318] "User" refers to an individual who uses the system, and in particular, is an entity that inputs information to create menus and purchase ingredients.

[0319] "Emotion recognition engine" refers to software or hardware technology that analyzes a user's facial expressions and vocal tone to generate emotional data for the user.

[0320] "Machine learning algorithms" refers to a set of computational methods and models for analyzing data and learning patterns, which are used here to generate meal suggestions.

[0321] A "generative AI model" refers to the entire system that uses machine learning algorithms to analyze user information and emotional data and generate optimal menu suggestions.

[0322] A "database" refers to a collection of information that accumulates information and emotional data input by users and is used in subsequent processing, as well as a system that manages this information.

[0323] "User terminal" refers to a device such as a smartphone or computer that allows a user to input information, display and select menu options, and check cooking instructions.

[0324] "Menu candidates" refer to multiple menu plans that the generative AI model suggests by analyzing the user's information and emotional data.

[0325] "Ingredient list" refers to a list of ingredients, seasonings, etc. required based on the menu options selected by the user.

[0326] "How to cook" refers to information that explains the steps to cook a dish based on a list of ingredients, and is provided in text or video format.

[0327] This system is intended to streamline user menu planning and ingredient purchasing, and to provide personalized meal suggestions based on emotions. Specific embodiments of this system will be described below.

[0328] Entering user information

[0329] User device:

[0330] Users open a dedicated application and log in to the system by entering the necessary authentication information on the login screen. After logging in, users enter information about breakfast, lunch, and dinner from the past month, the type of food they prefer (Japanese, Chinese, etc.), their desired daily calorie intake, and any food allergies they dislike. For example, they can enter information such as "toast and salad," "spaghetti," "gratin," "Japanese, Chinese," "1700 kcal / day," and "nuts, shrimp."

[0331] Receiving and storing data

[0332] server:

[0333] The server receives various information sent from the user's device. The received information is stored in a database and accumulated as profile information for each user. This database uses a relational database such as MySQL or PostgreSQL.

[0334] emotion recognition

[0335] User device:

[0336] Users input their daily physical condition, mood, and feedback on cooking into the application. The app also uses a camera and microphone with a built-in emotion recognition engine to measure the user's facial expressions and voice tone to generate emotion data. For example, if a user says, "I'm tired today," the emotion recognition engine analyzes this and recognizes it as "feeling tired."

[0337] server:

[0338] The server receives the emotion data sent from the user's device and stores it in a database.

[0339] Menu generation

[0340] server:

[0341] The server retrieves user information and emotional data from the database in parallel and executes a Python script to process them. The script performs analysis using machine learning algorithms (TensorFlow and PyTorch). As a result, it generates five types of meal suggestions that take into account nutritional balance, calorie intake, and emotional data. For example, if the user is determined to be "feeling tired," the suggestions will include meals using ingredients with a relaxing effect (herbal tea or warm soup).

[0342] Presentation and selection of menu options

[0343] server:

[0344] The server sends the five generated menu options to the user's device via API and reflects them in real time.

[0345] User device:

[0346] The user checks five menu options presented on the dedicated application screen and selects the most suitable option. For example, the user selects "Option 1."

[0347] Generate a materials list

[0348] server:

[0349] The server receives the user's selections and finalizes a monthly menu plan. Based on the finalized menu, it generates a list of ingredients needed, such as rice, miso, mackerel, and vegetables.

[0350] Providing cooking instructions

[0351] server:

[0352] The server retrieves cooking instructions for each dish in the confirmed menu from the database and sends them to the user's device. Cooking instructions are provided in text format and in the form of videos linked to external sites such as YouTube. For example, "How to make onigiri" includes text information such as "Step 1: Cook rice. Step 2: Shape the onigiri." as well as a video link explaining how to cook grilled salted mackerel.

[0353] User device:

[0354] Users can then review the provided ingredient list and cooking instructions, and use videos and links to help guide them through the cooking process.

[0355] Accumulating emotional data and reflecting it in the next proposal

[0356] server:

[0357] The server continuously accumulates user emotional data and uses it to propose the next menu. This data is analyzed by a machine learning algorithm, contributing to improving the accuracy of the entire system. Based on the user's past emotional data, it is possible to propose even more optimized menus.

[0358] Example prompt sentence:

[0359] "I'm tired today. Can you suggest a relaxing meal?"

[0360] In this way, the system utilizes the user's individual information and emotional data to create efficient and balanced menus and purchase ingredients.

[0361] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0362] Step 1:

[0363] Users open a dedicated application and enter the required authentication information on the login screen to log in to the system.

[0364] Input: Authentication information (user ID, password)

[0365] Output: Login success status

[0366] Specific operation: After successfully logging in, the user enters their dietary information for the past month, the type of food they want to eat, their daily calorie goal, and any food they dislike. For example, they can enter "toast and salad," "spaghetti," "gratin," "Japanese and Chinese food," "1700 kcal / day," and "nuts and shrimp."

[0367] Step 2:

[0368] The server receives parameters such as dietary information, preferences, and allergy information sent from the user terminal.

[0369] Input: User-entered information (past menus, desired type of food, calorie goal, disliked ingredients)

[0370] Output: User information stored in the database

[0371] Specific operation: The received information is stored in a database and accumulated as basic data for the system to make suggestions. This database is built using MySQL or PostgreSQL.

[0372] Step 3:

[0373] Users input their daily physical condition and emotional feedback through the application, and an emotion recognition engine analyzes the user's facial expressions and voice tone to generate emotional data.

[0374] Input: User's physical condition feedback, voice data, face image

[0375] Output: Emotion data (e.g., tiredness, stress, relaxation)

[0376] What it does: The user types "I'm tired today," and the system uses the camera and microphone to capture and analyze facial expressions and voice tone for sentiment analysis.

[0377] Step 4:

[0378] The server stores the emotion data received from the user's device in a database.

[0379] Input: Emotion data

[0380] Output: Emotion data stored in a database

[0381] Specific operation: The server receives emotion data in real time and stores it in a database along with existing user data.

[0382] Step 5:

[0383] The server retrieves stored user information and emotional data from the database and analyzes it using machine learning algorithms.

[0384] Input: User information, emotion data

[0385] Output: Menu suggestions (e.g., menus using ingredients with a relaxing effect)

[0386] How it works: Using TensorFlow and PyTorch, the system analyzes nutritional balance, calories, and emotional data to generate five different menu options. For example, it suggests relaxing herbal tea or warm soup to a user who is feeling tired.

[0387] Step 6:

[0388] The server sends the generated menu candidates to the user's terminal via an API.

[0389] Input: Generated menu candidates

[0390] Output: Menu suggestions displayed on the user's device

[0391] Specific operation: Data is sent from the server to the user's device via API, and menu suggestions are displayed in real time on the application screen.

[0392] Step 7:

[0393] The user reviews five possible menu options and selects the one they most desire.

[0394] Input: Menu candidate

[0395] Output: Selected menu

[0396] Specific operation: The user refers to the menu options displayed within the app and performs an operation such as selecting "Option 1."

[0397] Step 8:

[0398] The server receives the menu options selected by the user and finalizes the menu plan for one month.

[0399] Input: Selected menu

[0400] Output: Finalized menu and ingredients list

[0401] Specific operation: The server generates a list of necessary ingredients based on the selected menu, listing "rice, miso, mackerel, vegetables," etc.

[0402] Step 9:

[0403] Based on the confirmed menu, the server retrieves the recipes for each dish from the database and sends them to the user's terminal.

[0404] Input: Confirmed menu

[0405] Output: Recipe for each dish (text format, video format)

[0406] Specific operation: The server summarizes the cooking steps in text format and sends cooking steps such as "Step 1: Cook rice. Step 2: Shape the rice balls," as well as a video link explaining how to cook grilled salted mackerel.

[0407] Step 10:

[0408] Users check the provided ingredient list and cooking instructions and then actually create the dish.

[0409] Input: Ingredient list, recipe

[0410] Output: Finished dish

[0411] Specific actions: The user refers to the app and proceeds with cooking based on the provided instructions and videos. For example, watching the provided video, the user makes grilled salted mackerel.

[0412] Step 11:

[0413] The server continuously accumulates the user's emotional data and reflects it in the next menu suggestion.

[0414] Input: Daily emotion data

[0415] Output: Improve the accuracy of the next suggestion using accumulated emotion data

[0416] How it works: The server analyzes user sentiment data collected daily and uses it to make the next menu suggestion. Through continuous learning, the system can make more advanced suggestions.

[0417] Through the above processing steps, users can efficiently create personalized menus based on their emotions and physical condition, and easily prepare healthy and balanced meals.

[0418] (Application example 2)

[0419] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0420] The problem that this invention aims to solve is to provide healthy and balanced meals by reducing the effort and stress of daily menu planning and ingredient purchasing for users, and to provide a system that takes into account the user's emotional state, proposes personalized menus, and enables instant delivery orders.

[0421] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for the user to input past menu information, desired dish types, desired calorie intake, and disliked foods; means for receiving the input information and saving it in a database; AI means for analyzing the saved information and generating multiple menu candidates taking nutritional balance and calorie intake into consideration; means for transmitting the generated menu candidates to a user terminal; means for the user to select from the generated menu candidates; means for generating an ingredient list based on the menu selected by the user; means for providing the ingredient list and cooking instructions for each dish; emotion recognition means for recognizing the user's emotions and personalizing the menu based on the emotions; and ordering means for placing a delivery order based on the emotion-based recommended menu. This enables the provision of personalized, healthy, and balanced meals that take the user's emotions into consideration. Furthermore, the server allows for instant ordering through a delivery service, enhancing user convenience.

[0422] "User information" refers to data such as menu information, desired types of food, desired calorie intake, and disliked foods that the user has previously entered.

[0423] A "database" is a storage device that stores information received from users and uses it for subsequent processing.

[0424] "Nutritional balance" refers to the appropriate proportions and combinations of nutrients contained in food, and is important for maintaining a healthy diet.

[0425] "Calorie intake" refers to the amount of energy a user takes in per day, and is the energy value of a meal.

[0426] "Meal suggestions" are meal options suggested based on the user's preferences and health status.

[0427] "Emotion recognition means" refers to devices or technologies that analyze and recognize a user's emotional state from facial expressions, vocal tone, etc.

[0428] "Personalization" means optimizing experiences and services based on a user's individual preferences and feelings.

[0429] "Delivery Order" means the ordering process for the delivery of the meal selected by the User to the specified location.

[0430] "AI means" is a technology that uses machine learning and data analysis to process user information and generate optimal menus.

[0431] The "Ingredient List" is a list of ingredients required to prepare the selected menu.

[0432] "How-to" refers to the steps or methods for preparing a meal, and is an instruction provided in text or video format.

[0433] As part of the system, this invention reduces the effort required for users to plan daily meals and purchase ingredients, provides healthy and balanced meals, and even suggests personalized meals through emotion recognition. This system is implemented mainly through a user device such as a smartphone and a server.

[0434] Entering user information

[0435] Users access a dedicated application using their own smartphone or other device and log in to the system. Here, they input past menu information, the type of food they want (Japanese, Chinese, etc.), their desired calorie intake, and any food dislikes (including allergens). For example, a user might input "toast and salad" for breakfast, "spaghetti" for lunch, and "gratin" for dinner, select "Japanese, Chinese" as their preference, and input their calorie intake as "1700 kcal / day" and "nuts, shrimp" as their food dislikes. This information is sent from the user's device to the server and stored in a database.

[0436] Receiving and storing data

[0437] The server receives the information entered from the user's device and stores it in a database, which stores the user's personal information and preferences in the system for use in subsequent processing.

[0438] emotion recognition

[0439] Users input emotional information such as feedback on the cooking experience, daily physical condition, and mood. The app also incorporates a function to recognize emotions from the user's facial expressions and voice tone using emotion recognition techniques, such as Microsoft Azure Face API and OpenCV. This information is sent from the user's device to the server.

[0440] Menu generation

[0441] The server retrieves user information stored in the database and user emotion data recognized by the emotion recognition means. Based on this data, it analyzes it using a machine learning algorithm (for example, TensorFlow or Scikit-learn) to generate multiple menu candidates that take into account nutritional balance, calorie intake, and the user's emotions. For example, if the user is feeling stressed, it will suggest a menu using ingredients that have a relaxing effect.

[0442] Presentation and selection of menu options

[0443] The server sends the generated menu candidates to the user's terminal. The user reviews the presented menu candidates and selects the most suitable one. For example, the user selects "Candidate 1." This selection information is sent to the server.

[0444] Generate a materials list

[0445] The server receives the menu information selected by the user and generates a list of ingredients based on the selected menu, such as "rice, miso, mackerel, and vegetables."

[0446] Providing cooking instructions

[0447] The server provides cooking instructions for each dish according to the confirmed menu. This includes cooking instructions in text format and cooking videos. It also displays links to external cooking recipe sites. For example, the server provides text information such as "Step 1: Cook rice. Step 2: Shape the rice balls" for how to make rice balls, as well as a video explaining how to cook grilled salted mackerel.

[0448] Accumulating emotional data and reflecting it in the next proposal

[0449] The server accumulates the user's emotional data and reflects it in the next menu suggestion. Based on the user's past emotional data, it is possible to suggest a more optimized menu. By continuously learning from this data, the accuracy of the system improves.

[0450] Delivery Order

[0451] The system has a means for placing a delivery order based on the menu information selected by the user. For example, the system performs a procedure that seamlessly integrates the experience to order ingredients for batter from a delivery service based on the menu selected by the user.

[0452] Prompt Sentence Examples

[0453] Recommend and order the best meal based on the user's food preferences and emotional data. If the user is feeling stressed, suggest a relaxing dish. Create it based on the following information:

[0454] Past meals: Breakfast: "Toast", Lunch: "Salad", Dinner: "Curry"

[0455] Preferences: "Japanese food", "Chinese food"

[0456] Disliked foods: "Nuts", "Shrimp"

[0457] Today's mood: "I'm stressed"

[0458] Convert the suggested menu into a delivery order.

[0459] This allows us to provide personalized, healthy, and balanced meals that take into account the user's emotions, and also allows for instant ordering through delivery services, increasing user convenience.

[0460] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0461] Step 1:

[0462] The user inputs past menu information, desired dish types, desired calorie intake, and disliked foods into the user's device. This input information includes the user's past breakfast choices of "toast and salad," lunch choices of "spaghetti," and dinner choices of "gratin." The input information is sent to the server.

[0463] Step 2:

[0464] The server stores the information received from the user's device in a database, accumulating data on the user's preferences and nutritional intake for subsequent analysis. The input data is stored as is, and undergoes format conversion and field alignment.

[0465] Step 3:

[0466] The user terminal inputs emotional information such as daily physical condition and mood. An emotion recognition means is also used to recognize emotions from facial expressions and voice tones. The emotional information (e.g., "stress") is sent to the server.

[0467] Step 4:

[0468] The server uses a machine learning algorithm to analyze the emotion data acquired by the emotion recognition means and past menu information. This analysis generates multiple menu candidates that take into account nutritional balance and calorie intake. The input is emotion data and user preference data, and the output is the generated menu candidates.

[0469] Step 5:

[0470] The server sends the generated menu options to the user's device. For example, five menu options are displayed, each with its nutritional value and calorie count. The user can visually confirm these options.

[0471] Step 6:

[0472] The user terminal selects the most suitable menu from the presented menu candidates. This selection information is sent back to the server. The input is the selected menu, and the output is detailed information about the selected menu.

[0473] Step 7:

[0474] The server generates a list of ingredients based on the selected menu information. The input is the selected menu information, and the output is a list of ingredients (e.g., "rice, miso, mackerel, vegetables"). This list is checked against a database to define the required ingredients in detail.

[0475] Step 8:

[0476] The server provides recipes for each dish based on a confirmed menu. This includes text-based cooking instructions and video-based cooking instructions. The input is the menu information, and the output is links to the cooking instructions and videos.

[0477] Step 9:

[0478] The user device displays the provided ingredient list and cooking instructions, and then actually cooks the dish, allowing the user to cook with a high level of perfection.

[0479] Step 10:

[0480] The server accumulates the user's emotional data and menu selection history and reflects this in the next menu suggestion. The input is the user's emotional history and selection history, and the output is data that contributes to improving the accuracy of the next suggestion.

[0481] Step 11:

[0482] The user terminal places a delivery order based on the selected menu. This information is sent to the server, which then executes the delivery service. The input is the user's order information, and the output is order confirmation and delivery execution information.

[0483] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0484] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0485] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0486] [Second embodiment]

[0487] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0488] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0489] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0490] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0491] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0492] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0493] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0494] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0495] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0496] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0497] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0498] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0499] The purpose of this system is to reduce the effort and stress involved in daily menu planning and ingredient purchasing for users, and to provide healthy and balanced meals. The following describes the embodiments of the present invention.

[0500] Entering user information

[0501] User device:

[0502] Users access a dedicated application using their smartphone, computer, or other device and log in to the system. They input menu information from the past month, the type of food they prefer (Japanese, Chinese, etc.), their desired calorie intake, and any foods they dislike (including allergens). For example, a user might input "toast and salad" for breakfast, "spaghetti" for lunch, and "gratin" for dinner in the past. They might also select "Japanese, Chinese" as their preferences, specify their calorie intake as "1700 kcal / day," and input "nuts, shrimp" as foods they dislike.

[0503] Receiving and storing data

[0504] server:

[0505] The server receives the information sent from the user's device and stores it in a database, thereby retaining the individual user information needed for subsequent processing and analysis.

[0506] Menu generation

[0507] server:

[0508] The server retrieves the stored user information from the database. It then analyzes this information using a machine learning algorithm to generate the ideal menu for the next month, taking into account nutritional balance and calorie intake. Specifically, the AI ​​analyzes the user's past eating history and health status, and automatically generates five menu candidates. For example, one of the generated candidates might be "Breakfast: Rice balls and miso soup, Lunch: Fried rice, Dinner: Grilled salted mackerel."

[0509] Presentation and selection of menu options

[0510] server:

[0511] The server sends the five generated menu options to the user's terminal.

[0512] User device:

[0513] The user reviews the menu options and selects the most suitable one. For example, the user selects "Option 1."

[0514] Generate a materials list

[0515] server:

[0516] The server determines a monthly menu based on the menu selected by the user. A list of ingredients is generated based on the determined menu. For example, an ingredient list based on the determined menu may include "rice, miso, mackerel, and vegetables."

[0517] Providing cooking instructions

[0518] server:

[0519] The server provides instructions for each dish included in the confirmed menu. Detailed cooking instructions are prepared in the form of text and video, and links to major external recipe websites are also displayed. For example, when it comes to how to make rice balls, text information such as "Step 1: Cook rice. Step 2: Shape the rice balls" is provided, along with a video explaining how to cook grilled salted mackerel.

[0520] User device:

[0521] Users can check the provided ingredient list and cooking instructions, then actually cook the dish, and refer to videos and links to improve the quality of their cooking.

[0522] In this way, users can efficiently plan and shop for ingredients, and easily prepare balanced, healthy meals.

[0523] The processing flow will be explained below.

[0524] Step 1:

[0525] User device:

[0526] The user logs in to the system. After logging in, a special input form is displayed. Using this form, the user enters menu information for the past month. Specifically, the user enters information such as "January 1st: Breakfast - toast and salad, lunch - spaghetti, dinner - gratin."

[0527] Step 2:

[0528] User device:

[0529] The user selects the type of cuisine they desire. For example, they select "Japanese, Chinese." They also input their desired daily calorie intake. For example, they input "1700 kcal / day." They also input foods they dislike or are allergic to. For example, they input "nuts, shrimp."

[0530] Step 3:

[0531] server:

[0532] It receives information entered from the user's device and stores it in a database, which allows the system to store information about the user's personal information and preferences for subsequent processing.

[0533] Step 4:

[0534] server:

[0535] The system obtains the user's menu information and preferences for the past month from the database. Based on the obtained information, it uses a machine learning algorithm to perform an analysis. Based on this analysis, it generates five menu candidates that take into account nutritional balance and calorie intake. For example, candidate 1 would be "Breakfast: rice balls and miso soup, lunch: fried rice, dinner: grilled salted mackerel."

[0536] Step 5:

[0537] server:

[0538] The five generated menu options are sent to the user's device.

[0539] Step 6:

[0540] User device:

[0541] The user reviews the five menu options displayed. After reviewing the details of each option, the user can select the option that best suits them. For example, the user can select "Option 1."

[0542] Step 7:

[0543] server:

[0544] Receives menu information selected by the user and determines a monthly menu based on the selection. Based on the determined menu, generates a list of required ingredients. For example, the list may include "rice, miso, mackerel, and vegetables."

[0545] Step 8:

[0546] server:

[0547] It prepares recipes for each dish based on the confirmed menu, including text-based cooking instructions and video-based cooking instructions, and also generates links to major external recipe sites.

[0548] Step 9:

[0549] User device:

[0550] Users are provided with a list of ingredients and instructions for each dish, and are guided through videos and links to create the dish, making it easy for users to prepare a balanced meal.

[0551] This series of steps allows the system to significantly reduce the effort required for users to create menus and purchase ingredients, and supports healthy eating habits.

[0552] Example 1

[0553] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0554] In today's busy lifestyles, planning daily meals and purchasing ingredients can be a time-consuming and stressful process. It is also difficult and time-consuming to create a plan that takes nutritional balance into account. As a result, achieving a healthy and balanced diet can be challenging.

[0555] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0556] In this invention, the server includes a means for allowing a user to input past meal plan information, favorite types of dishes, desired energy intake, and ingredients to avoid, a means for receiving the input information and storing it in a data structure, and a generative AI model means for analyzing the stored information and generating multiple meal plan candidates taking into account nutritional balance and energy intake, thereby enabling a user to efficiently create menus, purchase necessary ingredients, and easily prepare balanced, healthy meals.

[0557] "User" refers to a person who uses the system to create menus and purchase ingredients.

[0558] "Means of input" refers to the mechanism by which users use a terminal to provide the system with information such as past meal planning information, preferred types of food, desired energy intake, and ingredients to avoid.

[0559] "Data Structure" refers to the arrangement of information within the system for organizing and temporarily or permanently storing information received from users.

[0560] "Generative AI model" refers to an artificial intelligence algorithm that generates meal plan suggestions that take into account nutritional balance and energy intake based on stored user information.

[0561] "Display Device" means a device for visually presenting generated meal plan suggestions and other information to a user, including, for example, a smartphone screen or a computer monitor.

[0562] "Ingredients List" refers to the list of ingredients needed to prepare a meal based on the meal plan selected by the User.

[0563] "Cooking instructions" refers to content that provides instructions and methods for preparing a dish using ingredients in text and video format.

[0564] "Energy intake" refers to the amount of calories a user needs per day.

[0565] The system of this invention allows users to efficiently plan menus and purchase ingredients, and easily prepare balanced, healthy meals. This system is implemented primarily using a user terminal and a server.

[0566] Entering user information

[0567] User terminal

[0568] Users log in to a dedicated application using a device such as a smartphone or computer. After logging in, users are prompted to enter the following information:

[0569] Menu information for the past month

[0570] For example: "Breakfast: Toast and salad", "Lunch: Spaghetti", "Dinner: Gratin"

[0571] Desired type of cuisine (Japanese, Chinese, etc.)

[0572] Example: "Japanese food, Chinese food"

[0573] Desired calorie intake

[0574] Example: "1700 kcal / day"

[0575] Foods you dislike (including allergens)

[0576] For example: "Nuts, shrimp."

[0577] Receiving and storing data

[0578] server

[0579] The information entered by the user is sent from the user terminal, and the server receives this information, which is then stored in a data structure for further processing.

[0580] Menu generation

[0581] server

[0582] The server retrieves the stored user information from the data structure and analyzes it using the generative AI model. Specifically, it performs the following steps:

[0583] Obtain the user's dietary history and health information.

[0584] Data is input into a generative AI model to generate multiple meal plan options that take into account nutritional balance and energy intake.

[0585] Example: "Breakfast: rice balls and miso soup, lunch: fried rice, dinner: grilled salted mackerel"

[0586] Presentation and selection of menu options

[0587] server

[0588] The server sends the generated meal plan candidates to the user terminal.

[0589] User terminal

[0590] The user checks the displayed menu options and selects the most suitable one. The selected information is sent to the server and used for the next process.

[0591] Generate a materials list

[0592] server

[0593] The server determines the menu for one month based on the selected menu and generates a list of ingredients, which is then sent to the user's device.

[0594] Example: "Rice, miso, mackerel, vegetables"

[0595] Providing cooking instructions

[0596] server

[0597] Based on the confirmed menu, the server provides cooking instructions for each dish, including text and video formats, as well as links to major external recipe sites.

[0598] Examples: "How to make onigiri: Step 1: Cook the rice. Step 2: Shape the onigiri" or "Video explaining how to cook grilled salted mackerel"

[0599] User terminal

[0600] Users can check the provided ingredient list and cooking instructions, then actually cook the dish, and refer to videos and links to improve the quality of their cooking.

[0601] In this way, the system helps users efficiently plan, purchase, and prepare balanced, healthy meals.

[0602] "Please enter the user's menu information for the past month. Example: Breakfast: Toast and salad, Lunch: Spaghetti, Dinner: Gratin"

[0603] "Please select the type of cuisine you would like. Example: Japanese, Chinese"

[0604] "Enter your desired daily calorie intake. Example: 1700 kcal / day"

[0605] "Please enter any foods you dislike or are allergic to. Example: nuts, shrimp."

[0606] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0607] Step 1: Enter your user information

[0608] User terminal

[0609] Users log in to a dedicated application using their smartphone or computer, and then enter the following information when prompted:

[0610] Menu information for the past month (e.g., "Breakfast: Toast and Salad," "Lunch: Spaghetti," "Dinner: Gratin")

[0611] Desired type of cuisine (e.g., "Japanese, Chinese")

[0612] Desired calorie intake (e.g., "1700 kcal / day")

[0613] Foods you dislike (e.g., "nuts, shrimp")

[0614] The input data sent to the server includes this user information.

[0615] Step 2: Receiving and storing data

[0616] server

[0617] The server receives the information sent from the user device. Specifically, when the input data reaches the server's API endpoint, the server performs data validation and format conversion, then executes a query and stores it in the database in the appropriate format.

[0618] Input: User's diet history, favorite dishes, desired calorie intake, ingredients to avoid

[0619] Output: Saved user information is recorded in the database

[0620] Step 3: Create a menu

[0621] server

[0622] The server retrieves stored user information from the database. Based on the retrieved data, it uses a generative AI model to generate multiple meal plan candidates that take into account nutritional balance and calorie intake. Specifically, the server performs the following steps:

[0623] Query the database to retrieve the user's dietary history and related information

[0624] Input the acquired data into the generative AI model and start the analysis

[0625] As a result of the analysis, five types of meal plan candidates are generated.

[0626] Example: "Breakfast: rice balls and miso soup, lunch: fried rice, dinner: grilled salted mackerel"

[0627] Input: User information stored in the database

[0628] Output: 5 meal plan candidates generated

[0629] Step 4: Presentation and selection of menu options

[0630] server

[0631] The generated meal plan candidates are sent to the user's device.

[0632] User terminal

[0633] The user reviews the proposed meal plans and selects the most suitable one. Specifically, the application displays a list of meal plan candidates, and when the user taps to select one, the selection information is sent to the server.

[0634] Input: 5 meal plan options

[0635] Output: The meal plan selected by the user

[0636] Step 5: Generate a materials list

[0637] server

[0638] The server determines a monthly menu based on the menu selected by the user. It then generates a list of ingredients required for this determined menu. Specifically, the server analyzes the menu information and lists the ingredients required for each dish. The list is then saved in a database and sent to the user's device.

[0639] Example: "Rice, miso, mackerel, vegetables"

[0640] Input: Menu information selected by the user

[0641] Output: Generated materials list

[0642] Step 6: Provide cooking instructions

[0643] server

[0644] The server provides cooking instructions for each dish based on the confirmed menu. This is provided in both text and video format, and also displays links to major cooking recipe sites. Specifically, it retrieves the relevant cooking instructions from a recipe database, generates video links and text data, and sends them to the user's device.

[0645] Examples: "How to make onigiri: Step 1: Cook the rice. Step 2: Shape the onigiri" or "Video explaining how to cook grilled salted mackerel"

[0646] User terminal

[0647] Users can check the provided ingredient list and cooking instructions, then actually create the dish, especially by referring to videos and links to improve the quality of the dish.

[0648] Input: Confirmed menu information

[0649] Output: Cooking instructions in text and video format

[0650] In this way, the system helps users efficiently plan meals, purchase the necessary ingredients, and prepare balanced, healthy meals.

[0651] (Application example 1)

[0652] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0653] The objective of this invention is to provide healthy and balanced meals by reducing the effort and stress involved in daily menu planning and ingredient purchasing for users. It also aims to provide a system that allows users to easily select menus, efficiently obtain the necessary ingredients, and prepare meals. In particular, it aims to realize a system that, by linking with food delivery services, saves users the trouble of purchasing ingredients on the spot.

[0654] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0655] In this invention, the server includes: means for a user to input past menu information, desired types of dishes, desired calorie intake, and disliked foods; means for receiving the input information and saving it in a database; AI means using a generative AI model that analyzes the saved information and generates multiple menu candidates taking into account nutritional balance and calorie intake; means for sending the generated menu candidates to a user terminal; means for the user to select from the generated menu candidates; means for generating a list of ingredients based on the menu selected by the user and linking it to a food delivery company; and means for providing the ingredient list and instructions for each dish. This allows users to easily plan healthy meals and reliably obtain the necessary ingredients.

[0656] "User device" refers to an electronic device such as a computer, smartphone, or tablet that a user uses to input information.

[0657] A "database" is a collection of electronic files that systematically stores information received from users and manages it so that it can be easily accessed and searched later.

[0658] A "generative AI model" is a component of artificial intelligence that uses machine learning algorithms to analyze user input and generate optimal menus.

[0659] "AI means" refers to systems or programs that use artificial intelligence technology to analyze data and perform specific tasks.

[0660] "Meal candidates" refer to multiple meal plans generated based on user input.

[0661] "Ingredient Delivery Service Provider" refers to a service provider that delivers the necessary ingredients according to the ingredients list based on the menu selected by the user.

[0662] This invention aims to provide users with healthy, balanced meals by reducing the hassle and stress of daily meal planning and ingredient purchasing. The system consists of a smartphone application, a cloud server, and a generative AI model. Each function of the system and its implementation are described below.

[0663] 1. Enter your user information

[0664] User device:

[0665] Users access the dedicated application using their smartphone or tablet and enter their past meal plan information, desired type of food, desired calorie intake, and disliked foods. Specifically, users enter their meal history from the past month, their preferred food category (e.g., Japanese food, Chinese food), their daily calorie intake (e.g., 1700 kcal / day), and allergy information (e.g., nuts, shrimp).

[0666] 2. Receipt and storage of data

[0667] server:

[0668] The server receives the information sent from the user's device and stores it in a database, thereby obtaining individual user information that can be used for subsequent data analysis.

[0669] 3. Menu generation

[0670] server:

[0671] The server retrieves the saved user information and analyzes it using a generative AI model. The generative AI model, which implements a machine learning algorithm (using TensorFlow or PyTorch, for example), generates multiple menu candidates that take into account past eating history, preferences, nutritional balance, and calorie intake. Examples of generated menu candidates include "rice balls and miso soup" for breakfast, "fried rice" for lunch, and "grilled salted mackerel" for dinner.

[0672] 4. Presentation and selection of menu options

[0673] server:

[0674] The five generated menu options are sent to the user's device.

[0675] User device:

[0676] The user reviews the proposed menu options and selects the most suitable one.

[0677] 5. Material list generation and automatic delivery linkage

[0678] server:

[0679] Based on the menu selected by the user, a one-month menu is confirmed and an ingredient list is generated according to the confirmed menu. The generated ingredient list is linked to a food delivery system on the cloud, and the necessary ingredients are automatically delivered to the user. The food delivery company periodically delivers the necessary ingredients to the user based on the generated ingredient list.

[0680] 6. Providing cooking instructions

[0681] server:

[0682] The server provides detailed recipes for each dish in the generated menu, with cooking instructions in text and video formats and links to external recipe sites.

[0683] User device:

[0684] Users check the provided list of ingredients and cooking instructions, and then actually cook the dish. For example, detailed instructions for making onigiri are displayed, such as "Step 1: Cook rice. Step 2: Shape the onigiri." Users can refer to videos and links to improve the quality of their cooking.

[0685] Examples and prompts

[0686] Examples:

[0687] The user inputs "I'm allergic to nuts and shrimp, and I like Japanese and Chinese food," and sets their calorie intake. The next month's menu is generated, consisting of "rice balls, miso soup, fried rice, and grilled salted mackerel," and the necessary ingredients are delivered by a food delivery company.

[0688] Example prompt sentence:

[0689] The user enters information about the past month's menu, favorite types of food, calorie intake, allergy information, etc. This information is used to create a balanced menu for the next month. Favorite types of food are "Japanese" and "Chinese," calorie intake is "1700 kcal / day," and allergy information is "nuts and shrimp." The generated menu should be "Breakfast: rice balls and miso soup, lunch: fried rice, dinner: grilled salted mackerel."

[0690] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0691] Step 1:

[0692] The user's device inputs past menu information, desired type of food, desired calorie intake, and disliked foods. The user then launches the application and inputs their meal history for the past month, their preferred food category (e.g., Japanese, Chinese), their daily calorie intake (e.g., 1700 kcal / day), and allergy information (e.g., nuts, shrimp). This information is temporarily stored on the user's device and sent for later processing.

[0693] Step 2:

[0694] The user device sends the input information to the server, which receives it and stores it in a database. This information includes the user's dietary history, preferences, allergy information, calorie intake, etc. The database is structured to allow efficient access and searching.

[0695] Step 3:

[0696] The server retrieves the stored user information from the database and analyzes it using a generative AI model. The server passes the retrieved data as input to the generative AI model (using TensorFlow and PyTorch). The model analyzes the user's past eating history and preferences, and generates optimal meal suggestions taking into account nutritional balance and calorie intake. Five types of meal suggestions are generated as output.

[0697] Step 4:

[0698] The server sends the five generated menu candidates to the user's device. The user's device receives this information and displays it on the screen. The user reviews the presented menu candidates and selects the most suitable one. The information on the selected menu is temporarily stored on the user's device and then sent back to the server.

[0699] Step 5:

[0700] The server determines a monthly menu based on the menu selected by the user. It then generates a list of ingredients required based on the determined menu. The ingredient list lists the ingredients required for each dish in the selected menu. The generated ingredient list is then sent to a food delivery system on the cloud.

[0701] Step 6:

[0702] The food delivery company prepares the necessary ingredients based on the ingredient list received from the server and creates a delivery schedule. The delivery company delivers the necessary ingredients periodically based on the user's address information. This ensures that the user can obtain the ingredients they need.

[0703] Step 7:

[0704] The server provides detailed instructions for each dish in the generated menu. The cooking instructions are prepared in text and video formats and displayed within the application. Links to external recipe websites are also displayed. Users can check this information and use it as a reference when actually cooking.

[0705] These are the specific processing steps of the system program that realizes the application example. At each step, it is clear what information is required as input and what data processing and calculations are performed to obtain the output.

[0706] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0707] This system aims to reduce the effort and stress involved in daily menu planning and ingredient purchasing for users, and to provide healthy, balanced meals. Furthermore, by combining it with an emotion engine, it can also provide personalized menu suggestions based on the user's emotions. Below, we will explain an embodiment of the present invention when combined with an emotion engine.

[0708] Entering user information

[0709] User device:

[0710] Users access a dedicated application using their own smartphone, computer, or other device and log in to the system. They enter menu information from the past month, the type of food they want (Japanese, Chinese, etc.), their desired calorie intake, and any foods they dislike (including allergens). For example, a user might enter "toast and salad" for breakfast, "spaghetti" for lunch, and "gratin" for dinner in the past. They might also select "Japanese, Chinese" as their preferences, specify their calorie intake as "1700 kcal / day," and enter "nuts, shrimp" as their disliked foods.

[0711] Receiving and storing data

[0712] server:

[0713] The server receives the information entered from the user's terminal and stores it in a database, which stores the user's personal information and preferences in the system for subsequent processing.

[0714] emotion recognition

[0715] User device:

[0716] Users input their feedback on the food, their daily physical condition, and their mood. The emotion engine also recognizes emotions from the user's facial expressions and tone of voice. This information is sent from the user's device to the server.

[0717] Menu generation

[0718] server:

[0719] The server retrieves the user's stored information from the database, as well as the user's emotional data recognized by the emotion engine. Based on this information, the system performs analysis using a machine learning algorithm. Based on this analysis, it generates five menu suggestions that take into account nutritional balance, calorie intake, and the user's emotions. For example, if the user is feeling stressed, it will suggest a menu using ingredients that have a relaxing effect.

[0720] Presentation and selection of menu options

[0721] server:

[0722] The server sends the five generated menu options to the user's terminal.

[0723] User device:

[0724] The user reviews the menu options and selects the most suitable one. For example, the user selects "Option 1."

[0725] Generate a materials list

[0726] server:

[0727] Receives menu information selected by the user and determines a monthly menu based on the selection. Generates a list of ingredients required based on the determined menu. For example, the list may include "rice, miso, mackerel, and vegetables."

[0728] Providing cooking instructions

[0729] server:

[0730] The server provides cooking instructions for each dish included in the confirmed menu. This includes cooking instructions in text format and cooking videos. It also displays links to major external cooking recipe sites. For example, the server provides text information such as "Step 1: Cook rice. Step 2: Shape the rice balls" for how to make rice balls, along with a video explaining how to cook grilled salted mackerel.

[0731] User device:

[0732] Users can check the provided ingredient list and cooking instructions, then actually cook the dish, and refer to videos and links to improve the quality of their cooking.

[0733] Accumulating emotional data and reflecting it in the next proposal

[0734] server:

[0735] The system accumulates user emotional data and reflects it in the next menu suggestion. Based on the user's past emotional data, it is possible to suggest more optimized menus. By continuously learning from this data, the system's accuracy improves.

[0736] In this way, the system can utilize the user's personal information and emotional data to efficiently plan meals and purchase ingredients, making it easy to prepare balanced, healthy meals.

[0737] The processing flow will be explained below.

[0738] Step 1:

[0739] User device:

[0740] The user logs in to the system. After logging in, a special input form is displayed. Using this form, the user enters menu information for the past month. Specifically, the user enters information such as "January 1st: Breakfast - toast and salad, lunch - spaghetti, dinner - gratin."

[0741] Step 2:

[0742] User device:

[0743] The user selects the type of cuisine they desire. For example, they select "Japanese, Chinese." They also input their desired daily calorie intake. For example, they input "1700 kcal / day." They also input foods they dislike or are allergic to. For example, they input "nuts, shrimp."

[0744] Step 3:

[0745] server:

[0746] It receives information entered from the user's device and stores it in a database, which allows the system to store information about the user's personal information and preferences for subsequent processing.

[0747] Step 4:

[0748] User device:

[0749] Users input their feedback, daily physical condition, and mood. For example, they can enter comments such as "I feel stressed today" or "I feel relaxed today." The emotion engine then analyzes the user's facial expressions and tone of voice to recognize their emotions. This information is sent from the user's device to the server.

[0750] Step 5:

[0751] server:

[0752] The system receives emotional data sent from the user's device and stores it in a database, allowing the user's emotional state to be accumulated and used for future suggestions.

[0753] Step 6:

[0754] server:

[0755] The system retrieves user information and emotional data stored in the database. Based on this information, it uses a machine learning algorithm to perform analysis. Based on this analysis, it generates five menu candidates that take into account nutritional balance, calorie intake, and the user's emotions. For example, if the user is feeling stressed, it will suggest a menu using ingredients that have a relaxing effect.

[0756] Step 7:

[0757] server:

[0758] The five generated menu options are sent to the user's device.

[0759] Step 8:

[0760] User device:

[0761] The user reviews the five menu options displayed. After reviewing the details of each option, the user can select the option that best suits them. For example, the user can select "Option 1."

[0762] Step 9:

[0763] server:

[0764] Receives menu information selected by the user and determines a monthly menu based on the selection. Generates a list of ingredients required based on the determined menu. For example, the list may include "rice, miso, mackerel, and vegetables."

[0765] Step 10:

[0766] server:

[0767] It prepares recipes for each dish based on the confirmed menu, including text-based cooking instructions and video-based cooking instructions, and also generates links to major external recipe sites.

[0768] Step 11:

[0769] User device:

[0770] Users are provided with a list of ingredients and instructions for each dish, and are guided through videos and links to create the dish, making it easy for users to prepare a balanced meal.

[0771] Step 12:

[0772] server:

[0773] The system accumulates user emotional data and reflects it in the next menu suggestion. Based on the user's past emotional data, it is possible to suggest more optimized menus. By continuously learning from this data, the system's accuracy improves.

[0774] Example 2

[0775] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0776] Conventional menu creation systems only consider the user's preferences and nutritional balance, and are unable to provide personalized suggestions that reflect the user's mood or physical condition at any given time. As a result, users spend a lot of time and effort trying to find meals that suit their mood and physical condition, which can cause mental stress. This issue is especially important in today's world, where healthy meal choices that take into account nutritional balance and calorie intake are required.

[0777] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0778] In this invention, the server includes: means for a user to input past menu information, desired types of dishes, desired calorie intake, and disliked foods; means for receiving the input information and saving it in a database; means for analyzing the user's facial expressions and voice tone using an emotion recognition engine to generate emotion data; generation AI model means equipped with a machine learning algorithm that analyzes the saved information and emotion data and generates multiple menu candidates taking into account nutritional balance, calorie intake, and the user's emotions; means for transmitting the generated menu candidates to a user terminal; means for the user to select from the generated menu candidates; means for generating an ingredient list based on the menu selected by the user; and means for providing the ingredient list and cooking instructions for each dish. This enables personalized menu suggestions that reflect the user's emotions and physical condition, allowing users to easily prepare healthy and balanced meals.

[0779] "User" refers to an individual who uses the system, and in particular, is an entity that inputs information to create menus and purchase ingredients.

[0780] "Emotion recognition engine" refers to software or hardware technology that analyzes a user's facial expressions and vocal tone to generate emotional data for the user.

[0781] "Machine learning algorithms" refers to a set of computational methods and models for analyzing data and learning patterns, which are used here to generate meal suggestions.

[0782] A "generative AI model" refers to the entire system that uses machine learning algorithms to analyze user information and emotional data and generate optimal menu suggestions.

[0783] A "database" refers to a collection of information that accumulates information and emotional data input by users and is used in subsequent processing, as well as a system that manages this information.

[0784] "User terminal" refers to a device such as a smartphone or computer that allows a user to input information, display and select menu options, and check cooking instructions.

[0785] "Menu candidates" refer to multiple menu plans that the generative AI model suggests by analyzing the user's information and emotional data.

[0786] "Ingredient list" refers to a list of ingredients, seasonings, etc. required based on the menu options selected by the user.

[0787] "How to cook" refers to information that explains the steps to cook a dish based on a list of ingredients, and is provided in text or video format.

[0788] This system is intended to streamline user menu planning and ingredient purchasing, and to provide personalized meal suggestions based on emotions. Specific embodiments of this system will be described below.

[0789] Entering user information

[0790] User device:

[0791] Users open a dedicated application and log in to the system by entering the necessary authentication information on the login screen. After logging in, users enter information about breakfast, lunch, and dinner from the past month, the type of food they prefer (Japanese, Chinese, etc.), their desired daily calorie intake, and any food allergies they dislike. For example, they can enter information such as "toast and salad," "spaghetti," "gratin," "Japanese, Chinese," "1700 kcal / day," and "nuts, shrimp."

[0792] Receiving and storing data

[0793] server:

[0794] The server receives various information sent from the user's device. The received information is stored in a database and accumulated as profile information for each user. This database uses a relational database such as MySQL or PostgreSQL.

[0795] emotion recognition

[0796] User device:

[0797] Users input their daily physical condition, mood, and feedback on cooking into the application. The app also uses a camera and microphone with a built-in emotion recognition engine to measure the user's facial expressions and voice tone to generate emotion data. For example, if a user says, "I'm tired today," the emotion recognition engine analyzes this and recognizes it as "feeling tired."

[0798] server:

[0799] The server receives the emotion data sent from the user's device and stores it in a database.

[0800] Menu generation

[0801] server:

[0802] The server retrieves user information and emotional data from the database in parallel and executes a Python script to process them. The script performs analysis using machine learning algorithms (TensorFlow and PyTorch). As a result, it generates five types of meal suggestions that take into account nutritional balance, calorie intake, and emotional data. For example, if the user is determined to be "feeling tired," the suggestions will include meals using ingredients with a relaxing effect (herbal tea or warm soup).

[0803] Presentation and selection of menu options

[0804] server:

[0805] The server sends the five generated menu options to the user's device via API and reflects them in real time.

[0806] User device:

[0807] The user checks five menu options presented on the dedicated application screen and selects the most suitable option. For example, the user selects "Option 1."

[0808] Generate a materials list

[0809] server:

[0810] The server receives the user's selections and finalizes a monthly menu plan. Based on the finalized menu, it generates a list of ingredients needed, such as rice, miso, mackerel, and vegetables.

[0811] Providing cooking instructions

[0812] server:

[0813] The server retrieves cooking instructions for each dish in the confirmed menu from the database and sends them to the user's device. Cooking instructions are provided in text format and in the form of videos linked to external sites such as YouTube. For example, "How to make onigiri" includes text information such as "Step 1: Cook rice. Step 2: Shape the onigiri." as well as a video link explaining how to cook grilled salted mackerel.

[0814] User device:

[0815] Users can then review the provided ingredient list and cooking instructions, and use videos and links to help guide them through the cooking process.

[0816] Accumulating emotional data and reflecting it in the next proposal

[0817] server:

[0818] The server continuously accumulates user emotional data and uses it to propose the next menu. This data is analyzed by a machine learning algorithm, contributing to improving the accuracy of the entire system. Based on the user's past emotional data, it is possible to propose even more optimized menus.

[0819] Example prompt sentence:

[0820] "I'm tired today. Can you suggest a relaxing meal?"

[0821] In this way, the system utilizes the user's individual information and emotional data to create efficient and balanced menus and purchase ingredients.

[0822] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0823] Step 1:

[0824] Users open a dedicated application and enter the required authentication information on the login screen to log in to the system.

[0825] Input: Authentication information (user ID, password)

[0826] Output: Login success status

[0827] Specific operation: After successfully logging in, the user enters their dietary information for the past month, the type of food they want to eat, their daily calorie goal, and any food they dislike. For example, they can enter "toast and salad," "spaghetti," "gratin," "Japanese and Chinese food," "1700 kcal / day," and "nuts and shrimp."

[0828] Step 2:

[0829] The server receives parameters such as dietary information, preferences, and allergy information sent from the user terminal.

[0830] Input: User-entered information (past menus, desired type of food, calorie goal, disliked ingredients)

[0831] Output: User information stored in the database

[0832] Specific operation: The received information is stored in a database and accumulated as basic data for the system to make suggestions. This database is built using MySQL or PostgreSQL.

[0833] Step 3:

[0834] Users input their daily physical condition and emotional feedback through the application, and an emotion recognition engine analyzes the user's facial expressions and voice tone to generate emotional data.

[0835] Input: User's physical condition feedback, voice data, face image

[0836] Output: Emotion data (e.g., tiredness, stress, relaxation)

[0837] What it does: The user types "I'm tired today," and the system uses the camera and microphone to capture and analyze facial expressions and voice tone for sentiment analysis.

[0838] Step 4:

[0839] The server stores the emotion data received from the user's device in a database.

[0840] Input: Emotion data

[0841] Output: Emotion data stored in a database

[0842] Specific operation: The server receives emotion data in real time and stores it in a database along with existing user data.

[0843] Step 5:

[0844] The server retrieves stored user information and emotional data from the database and analyzes it using machine learning algorithms.

[0845] Input: User information, emotion data

[0846] Output: Menu suggestions (e.g., menus using ingredients with a relaxing effect)

[0847] How it works: Using TensorFlow and PyTorch, the system analyzes nutritional balance, calories, and emotional data to generate five different menu options. For example, it suggests relaxing herbal tea or warm soup to a user who is feeling tired.

[0848] Step 6:

[0849] The server sends the generated menu candidates to the user's terminal via an API.

[0850] Input: Generated menu candidates

[0851] Output: Menu suggestions displayed on the user's device

[0852] Specific operation: Data is sent from the server to the user's device via API, and menu suggestions are displayed in real time on the application screen.

[0853] Step 7:

[0854] The user reviews five possible menu options and selects the one they most desire.

[0855] Input: Menu candidate

[0856] Output: Selected menu

[0857] Specific operation: The user refers to the menu options displayed within the app and performs an operation such as selecting "Option 1."

[0858] Step 8:

[0859] The server receives the menu options selected by the user and finalizes the menu plan for one month.

[0860] Input: Selected menu

[0861] Output: Finalized menu and ingredients list

[0862] Specific operation: The server generates a list of necessary ingredients based on the selected menu, listing "rice, miso, mackerel, vegetables," etc.

[0863] Step 9:

[0864] Based on the confirmed menu, the server retrieves the recipes for each dish from the database and sends them to the user's terminal.

[0865] Input: Confirmed menu

[0866] Output: Recipe for each dish (text format, video format)

[0867] Specific operation: The server summarizes the cooking steps in text format and sends cooking steps such as "Step 1: Cook rice. Step 2: Shape the rice balls," as well as a video link explaining how to cook grilled salted mackerel.

[0868] Step 10:

[0869] Users check the provided ingredient list and cooking instructions and then actually create the dish.

[0870] Input: Ingredient list, recipe

[0871] Output: Finished dish

[0872] Specific actions: The user refers to the app and proceeds with cooking based on the provided instructions and videos. For example, watching the provided video, the user makes grilled salted mackerel.

[0873] Step 11:

[0874] The server continuously accumulates the user's emotional data and reflects it in the next menu suggestion.

[0875] Input: Daily emotion data

[0876] Output: Improve the accuracy of the next suggestion using accumulated emotion data

[0877] How it works: The server analyzes user sentiment data collected daily and uses it to make the next menu suggestion. Through continuous learning, the system can make more advanced suggestions.

[0878] Through the above processing steps, users can efficiently create personalized menus based on their emotions and physical condition, and easily prepare healthy and balanced meals.

[0879] (Application example 2)

[0880] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0881] The problem that this invention aims to solve is to provide healthy and balanced meals by reducing the effort and stress of daily menu planning and ingredient purchasing for users, and to provide a system that takes into account the user's emotional state, proposes personalized menus, and enables instant delivery orders.

[0882] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for the user to input past menu information, desired dish types, desired calorie intake, and disliked foods; means for receiving the input information and saving it in a database; AI means for analyzing the saved information and generating multiple menu candidates taking nutritional balance and calorie intake into consideration; means for transmitting the generated menu candidates to a user terminal; means for the user to select from the generated menu candidates; means for generating an ingredient list based on the menu selected by the user; means for providing the ingredient list and cooking instructions for each dish; emotion recognition means for recognizing the user's emotions and personalizing the menu based on the emotions; and ordering means for placing a delivery order based on the emotion-based recommended menu. This enables the provision of personalized, healthy, and balanced meals that take the user's emotions into consideration. Furthermore, the server allows for instant ordering through a delivery service, enhancing user convenience.

[0883] "User information" refers to data such as menu information, desired types of food, desired calorie intake, and disliked foods that the user has previously entered.

[0884] A "database" is a storage device that stores information received from users and uses it for subsequent processing.

[0885] "Nutritional balance" refers to the appropriate proportions and combinations of nutrients contained in food, and is important for maintaining a healthy diet.

[0886] "Calorie intake" refers to the amount of energy a user takes in per day, and is the energy value of a meal.

[0887] "Meal suggestions" are meal options suggested based on the user's preferences and health status.

[0888] "Emotion recognition means" refers to devices or technologies that analyze and recognize a user's emotional state from facial expressions, vocal tone, etc.

[0889] "Personalization" means optimizing experiences and services based on a user's individual preferences and feelings.

[0890] "Delivery Order" means the ordering process for the delivery of the meal selected by the User to the specified location.

[0891] "AI means" is a technology that uses machine learning and data analysis to process user information and generate optimal menus.

[0892] The "Ingredient List" is a list of ingredients required to prepare the selected menu.

[0893] "How-to" refers to the steps or methods for preparing a meal, and is an instruction provided in text or video format.

[0894] As part of the system, this invention reduces the effort required for users to plan daily meals and purchase ingredients, provides healthy and balanced meals, and even suggests personalized meals through emotion recognition. This system is implemented mainly through a user device such as a smartphone and a server.

[0895] Entering user information

[0896] Users access a dedicated application using their own smartphone or other device and log in to the system. Here, they input past menu information, the type of food they want (Japanese, Chinese, etc.), their desired calorie intake, and any food dislikes (including allergens). For example, a user might input "toast and salad" for breakfast, "spaghetti" for lunch, and "gratin" for dinner, select "Japanese, Chinese" as their preference, and input their calorie intake as "1700 kcal / day" and "nuts, shrimp" as their food dislikes. This information is sent from the user's device to the server and stored in a database.

[0897] Receiving and storing data

[0898] The server receives the information entered from the user's device and stores it in a database, which stores the user's personal information and preferences in the system for use in subsequent processing.

[0899] emotion recognition

[0900] Users input emotional information such as feedback on the cooking experience, daily physical condition, and mood. The app also incorporates a function to recognize emotions from the user's facial expressions and voice tone using emotion recognition techniques, such as Microsoft Azure Face API and OpenCV. This information is sent from the user's device to the server.

[0901] Menu generation

[0902] The server retrieves user information stored in the database and user emotion data recognized by the emotion recognition means. Based on this data, it analyzes it using a machine learning algorithm (for example, TensorFlow or Scikit-learn) to generate multiple menu candidates that take into account nutritional balance, calorie intake, and the user's emotions. For example, if the user is feeling stressed, it will suggest a menu using ingredients that have a relaxing effect.

[0903] Presentation and selection of menu options

[0904] The server sends the generated menu candidates to the user's terminal. The user reviews the presented menu candidates and selects the most suitable one. For example, the user selects "Candidate 1." This selection information is sent to the server.

[0905] Generate a materials list

[0906] The server receives the menu information selected by the user and generates a list of ingredients based on the selected menu, such as "rice, miso, mackerel, and vegetables."

[0907] Providing cooking instructions

[0908] The server provides cooking instructions for each dish according to the confirmed menu. This includes cooking instructions in text format and cooking videos. It also displays links to external cooking recipe sites. For example, the server provides text information such as "Step 1: Cook rice. Step 2: Shape the rice balls" for how to make rice balls, as well as a video explaining how to cook grilled salted mackerel.

[0909] Accumulating emotional data and reflecting it in the next proposal

[0910] The server accumulates the user's emotional data and reflects it in the next menu suggestion. Based on the user's past emotional data, it is possible to suggest a more optimized menu. By continuously learning from this data, the accuracy of the system improves.

[0911] Delivery Order

[0912] The system has a means for placing a delivery order based on the menu information selected by the user. For example, the system performs a procedure that seamlessly integrates the experience to order ingredients for batter from a delivery service based on the menu selected by the user.

[0913] Prompt Sentence Examples

[0914] Recommend and order the best meal based on the user's food preferences and emotional data. If the user is feeling stressed, suggest a relaxing dish. Create it based on the following information:

[0915] Past meals: Breakfast: "Toast", Lunch: "Salad", Dinner: "Curry"

[0916] Preferences: "Japanese food", "Chinese food"

[0917] Disliked foods: "Nuts", "Shrimp"

[0918] Today's mood: "I'm stressed"

[0919] Convert the suggested menu into a delivery order.

[0920] This allows us to provide personalized, healthy, and balanced meals that take into account the user's emotions, and also allows for instant ordering through delivery services, increasing user convenience.

[0921] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0922] Step 1:

[0923] The user inputs past menu information, desired dish types, desired calorie intake, and disliked foods into the user's device. This input information includes the user's past breakfast choices of "toast and salad," lunch choices of "spaghetti," and dinner choices of "gratin." The input information is sent to the server.

[0924] Step 2:

[0925] The server stores the information received from the user's device in a database, accumulating data on the user's preferences and nutritional intake for subsequent analysis. The input data is stored as is, and undergoes format conversion and field alignment.

[0926] Step 3:

[0927] The user terminal inputs emotional information such as daily physical condition and mood. An emotion recognition means is also used to recognize emotions from facial expressions and voice tones. The emotional information (e.g., "stress") is sent to the server.

[0928] Step 4:

[0929] The server uses a machine learning algorithm to analyze the emotion data acquired by the emotion recognition means and past menu information. This analysis generates multiple menu candidates that take into account nutritional balance and calorie intake. The input is emotion data and user preference data, and the output is the generated menu candidates.

[0930] Step 5:

[0931] The server sends the generated menu options to the user's device. For example, five menu options are displayed, each with its nutritional value and calorie count. The user can visually confirm these options.

[0932] Step 6:

[0933] The user terminal selects the most suitable menu from the presented menu candidates. This selection information is sent back to the server. The input is the selected menu, and the output is detailed information about the selected menu.

[0934] Step 7:

[0935] The server generates a list of ingredients based on the selected menu information. The input is the selected menu information, and the output is a list of ingredients (e.g., "rice, miso, mackerel, vegetables"). This list is checked against a database to define the required ingredients in detail.

[0936] Step 8:

[0937] The server provides recipes for each dish based on a confirmed menu. This includes text-based cooking instructions and video-based cooking instructions. The input is the menu information, and the output is links to the cooking instructions and videos.

[0938] Step 9:

[0939] The user device displays the provided ingredient list and cooking instructions, and then actually cooks the dish, allowing the user to cook with a high level of perfection.

[0940] Step 10:

[0941] The server accumulates the user's emotional data and menu selection history and reflects this in the next menu suggestion. The input is the user's emotional history and selection history, and the output is data that contributes to improving the accuracy of the next suggestion.

[0942] Step 11:

[0943] The user terminal places a delivery order based on the selected menu. This information is sent to the server, which then executes the delivery service. The input is the user's order information, and the output is order confirmation and delivery execution information.

[0944] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0945] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0946] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0947] [Third embodiment]

[0948] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0949] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0950] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0951] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0952] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0953] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0954] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0955] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0956] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0957] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0958] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0959] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0960] The purpose of this system is to reduce the effort and stress involved in daily menu planning and ingredient purchasing for users, and to provide healthy and balanced meals. The following describes the embodiments of the present invention.

[0961] Entering user information

[0962] User device:

[0963] Users access a dedicated application using their smartphone, computer, or other device and log in to the system. They input menu information from the past month, the type of food they prefer (Japanese, Chinese, etc.), their desired calorie intake, and any foods they dislike (including allergens). For example, a user might input "toast and salad" for breakfast, "spaghetti" for lunch, and "gratin" for dinner in the past. They might also select "Japanese, Chinese" as their preferences, specify their calorie intake as "1700 kcal / day," and input "nuts, shrimp" as foods they dislike.

[0964] Receiving and storing data

[0965] server:

[0966] The server receives the information sent from the user's device and stores it in a database, thereby retaining the individual user information needed for subsequent processing and analysis.

[0967] Menu generation

[0968] server:

[0969] The server retrieves the stored user information from the database. It then analyzes this information using a machine learning algorithm to generate the ideal menu for the next month, taking into account nutritional balance and calorie intake. Specifically, the AI ​​analyzes the user's past eating history and health status, and automatically generates five menu candidates. For example, one of the generated candidates might be "Breakfast: Rice balls and miso soup, Lunch: Fried rice, Dinner: Grilled salted mackerel."

[0970] Presentation and selection of menu options

[0971] server:

[0972] The server sends the five generated menu options to the user's terminal.

[0973] User device:

[0974] The user reviews the menu options and selects the most suitable one. For example, the user selects "Option 1."

[0975] Generate a materials list

[0976] server:

[0977] The server determines a monthly menu based on the menu selected by the user. A list of ingredients is generated based on the determined menu. For example, an ingredient list based on the determined menu may include "rice, miso, mackerel, and vegetables."

[0978] Providing cooking instructions

[0979] server:

[0980] The server provides instructions for each dish included in the confirmed menu. Detailed cooking instructions are prepared in the form of text and video, and links to major external recipe websites are also displayed. For example, when it comes to how to make rice balls, text information such as "Step 1: Cook rice. Step 2: Shape the rice balls" is provided, along with a video explaining how to cook grilled salted mackerel.

[0981] User device:

[0982] Users can check the provided ingredient list and cooking instructions, then actually cook the dish, and refer to videos and links to improve the quality of their cooking.

[0983] In this way, users can efficiently plan and shop for ingredients, and easily prepare balanced, healthy meals.

[0984] The processing flow will be explained below.

[0985] Step 1:

[0986] User device:

[0987] The user logs in to the system. After logging in, a special input form is displayed. Using this form, the user enters menu information for the past month. Specifically, the user enters information such as "January 1st: Breakfast - toast and salad, lunch - spaghetti, dinner - gratin."

[0988] Step 2:

[0989] User device:

[0990] The user selects the type of cuisine they desire. For example, they select "Japanese, Chinese." They also input their desired daily calorie intake. For example, they input "1700 kcal / day." They also input foods they dislike or are allergic to. For example, they input "nuts, shrimp."

[0991] Step 3:

[0992] server:

[0993] It receives information entered from the user's device and stores it in a database, which allows the system to store information about the user's personal information and preferences for subsequent processing.

[0994] Step 4:

[0995] server:

[0996] The system obtains the user's menu information and preferences for the past month from the database. Based on the obtained information, it uses a machine learning algorithm to perform an analysis. Based on this analysis, it generates five menu candidates that take into account nutritional balance and calorie intake. For example, candidate 1 would be "Breakfast: rice balls and miso soup, lunch: fried rice, dinner: grilled salted mackerel."

[0997] Step 5:

[0998] server:

[0999] The five generated menu options are sent to the user's device.

[1000] Step 6:

[1001] User device:

[1002] The user reviews the five menu options displayed. After reviewing the details of each option, the user can select the option that best suits them. For example, the user can select "Option 1."

[1003] Step 7:

[1004] server:

[1005] Receives menu information selected by the user and determines a monthly menu based on the selection. Based on the determined menu, generates a list of required ingredients. For example, the list may include "rice, miso, mackerel, and vegetables."

[1006] Step 8:

[1007] server:

[1008] It prepares recipes for each dish based on the confirmed menu, including text-based cooking instructions and video-based cooking instructions, and also generates links to major external recipe sites.

[1009] Step 9:

[1010] User device:

[1011] Users are provided with a list of ingredients and instructions for each dish, and are guided through videos and links to create the dish, making it easy for users to prepare a balanced meal.

[1012] This series of steps allows the system to significantly reduce the effort required for users to create menus and purchase ingredients, and supports healthy eating habits.

[1013] Example 1

[1014] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1015] In today's busy lifestyles, planning daily meals and purchasing ingredients can be a time-consuming and stressful process. It is also difficult and time-consuming to create a plan that takes nutritional balance into account. As a result, achieving a healthy and balanced diet can be challenging.

[1016] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1017] In this invention, the server includes a means for allowing a user to input past meal plan information, favorite types of dishes, desired energy intake, and ingredients to avoid, a means for receiving the input information and storing it in a data structure, and a generative AI model means for analyzing the stored information and generating multiple meal plan candidates taking into account nutritional balance and energy intake, thereby enabling a user to efficiently create menus, purchase necessary ingredients, and easily prepare balanced, healthy meals.

[1018] "User" refers to a person who uses the system to create menus and purchase ingredients.

[1019] "Means of input" refers to the mechanism by which users use a terminal to provide the system with information such as past meal planning information, preferred types of food, desired energy intake, and ingredients to avoid.

[1020] "Data Structure" refers to the arrangement of information within the system for organizing and temporarily or permanently storing information received from users.

[1021] "Generative AI model" refers to an artificial intelligence algorithm that generates meal plan suggestions that take into account nutritional balance and energy intake based on stored user information.

[1022] "Display Device" means a device for visually presenting generated meal plan suggestions and other information to a user, including, for example, a smartphone screen or a computer monitor.

[1023] "Ingredients List" refers to the list of ingredients needed to prepare a meal based on the meal plan selected by the User.

[1024] "Cooking instructions" refers to content that provides instructions and methods for preparing a dish using ingredients in text and video format.

[1025] "Energy intake" refers to the amount of calories a user needs per day.

[1026] The system of this invention allows users to efficiently plan menus and purchase ingredients, and easily prepare balanced, healthy meals. This system is implemented primarily using a user terminal and a server.

[1027] Entering user information

[1028] User terminal

[1029] Users log in to a dedicated application using a device such as a smartphone or computer. After logging in, users are prompted to enter the following information:

[1030] Menu information for the past month

[1031] For example: "Breakfast: Toast and salad", "Lunch: Spaghetti", "Dinner: Gratin"

[1032] Desired type of cuisine (Japanese, Chinese, etc.)

[1033] Example: "Japanese food, Chinese food"

[1034] Desired calorie intake

[1035] Example: "1700 kcal / day"

[1036] Foods you dislike (including allergens)

[1037] For example: "Nuts, shrimp."

[1038] Receiving and storing data

[1039] server

[1040] The information entered by the user is sent from the user terminal, and the server receives this information, which is then stored in a data structure for further processing.

[1041] Menu generation

[1042] server

[1043] The server retrieves the stored user information from the data structure and analyzes it using the generative AI model. Specifically, it performs the following steps:

[1044] Obtain the user's dietary history and health information.

[1045] Data is input into a generative AI model to generate multiple meal plan options that take into account nutritional balance and energy intake.

[1046] Example: "Breakfast: rice balls and miso soup, lunch: fried rice, dinner: grilled salted mackerel"

[1047] Presentation and selection of menu options

[1048] server

[1049] The server sends the generated meal plan candidates to the user terminal.

[1050] User terminal

[1051] The user checks the displayed menu options and selects the most suitable one. The selected information is sent to the server and used for the next process.

[1052] Generate a materials list

[1053] server

[1054] The server determines the menu for one month based on the selected menu and generates a list of ingredients, which is then sent to the user's device.

[1055] Example: "Rice, miso, mackerel, vegetables"

[1056] Providing cooking instructions

[1057] server

[1058] Based on the confirmed menu, the server provides cooking instructions for each dish, including text and video formats, as well as links to major external recipe sites.

[1059] Examples: "How to make onigiri: Step 1: Cook the rice. Step 2: Shape the onigiri" or "Video explaining how to cook grilled salted mackerel"

[1060] User terminal

[1061] Users can check the provided ingredient list and cooking instructions, then actually cook the dish, and refer to videos and links to improve the quality of their cooking.

[1062] In this way, the system helps users efficiently plan, purchase, and prepare balanced, healthy meals.

[1063] "Please enter the user's menu information for the past month. Example: Breakfast: Toast and salad, Lunch: Spaghetti, Dinner: Gratin"

[1064] "Please select the type of cuisine you would like. Example: Japanese, Chinese"

[1065] "Enter your desired daily calorie intake. Example: 1700 kcal / day"

[1066] "Please enter any foods you dislike or are allergic to. Example: nuts, shrimp."

[1067] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1068] Step 1: Enter your user information

[1069] User terminal

[1070] Users log in to a dedicated application using their smartphone or computer, and then enter the following information when prompted:

[1071] Menu information for the past month (e.g., "Breakfast: Toast and Salad," "Lunch: Spaghetti," "Dinner: Gratin")

[1072] Desired type of cuisine (e.g., "Japanese, Chinese")

[1073] Desired calorie intake (e.g., "1700 kcal / day")

[1074] Foods you dislike (e.g., "nuts, shrimp")

[1075] The input data sent to the server includes this user information.

[1076] Step 2: Receiving and storing data

[1077] server

[1078] The server receives the information sent from the user device. Specifically, when the input data reaches the server's API endpoint, the server performs data validation and format conversion, then executes a query and stores it in the database in the appropriate format.

[1079] Input: User's diet history, favorite dishes, desired calorie intake, ingredients to avoid

[1080] Output: Saved user information is recorded in the database

[1081] Step 3: Create a menu

[1082] server

[1083] The server retrieves stored user information from the database. Based on the retrieved data, it uses a generative AI model to generate multiple meal plan candidates that take into account nutritional balance and calorie intake. Specifically, the server performs the following steps:

[1084] Query the database to retrieve the user's dietary history and related information

[1085] Input the acquired data into the generative AI model and start the analysis

[1086] As a result of the analysis, five types of meal plan candidates are generated.

[1087] Example: "Breakfast: rice balls and miso soup, lunch: fried rice, dinner: grilled salted mackerel"

[1088] Input: User information stored in the database

[1089] Output: 5 meal plan candidates generated

[1090] Step 4: Presentation and selection of menu options

[1091] server

[1092] The generated meal plan candidates are sent to the user's device.

[1093] User terminal

[1094] The user reviews the proposed meal plans and selects the most suitable one. Specifically, the application displays a list of meal plan candidates, and when the user taps to select one, the selection information is sent to the server.

[1095] Input: 5 meal plan options

[1096] Output: The meal plan selected by the user

[1097] Step 5: Generate a materials list

[1098] server

[1099] The server determines a monthly menu based on the menu selected by the user. It then generates a list of ingredients required for this determined menu. Specifically, the server analyzes the menu information and lists the ingredients required for each dish. The list is then saved in a database and sent to the user's device.

[1100] Example: "Rice, miso, mackerel, vegetables"

[1101] Input: Menu information selected by the user

[1102] Output: Generated materials list

[1103] Step 6: Provide cooking instructions

[1104] server

[1105] The server provides cooking instructions for each dish based on the confirmed menu. This is provided in both text and video format, and also displays links to major cooking recipe sites. Specifically, it retrieves the relevant cooking instructions from a recipe database, generates video links and text data, and sends them to the user's device.

[1106] Examples: "How to make onigiri: Step 1: Cook the rice. Step 2: Shape the onigiri" or "Video explaining how to cook grilled salted mackerel"

[1107] User terminal

[1108] Users can check the provided ingredient list and cooking instructions, then actually create the dish, especially by referring to videos and links to improve the quality of the dish.

[1109] Input: Confirmed menu information

[1110] Output: Cooking instructions in text and video format

[1111] In this way, the system helps users efficiently plan meals, purchase the necessary ingredients, and prepare balanced, healthy meals.

[1112] (Application example 1)

[1113] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1114] The objective of this invention is to provide healthy and balanced meals by reducing the effort and stress involved in daily menu planning and ingredient purchasing for users. It also aims to provide a system that allows users to easily select menus, efficiently obtain the necessary ingredients, and prepare meals. In particular, it aims to realize a system that, by linking with food delivery services, saves users the trouble of purchasing ingredients on the spot.

[1115] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1116] In this invention, the server includes: means for a user to input past menu information, desired types of dishes, desired calorie intake, and disliked foods; means for receiving the input information and saving it in a database; AI means using a generative AI model that analyzes the saved information and generates multiple menu candidates taking into account nutritional balance and calorie intake; means for sending the generated menu candidates to a user terminal; means for the user to select from the generated menu candidates; means for generating a list of ingredients based on the menu selected by the user and linking it to a food delivery company; and means for providing the ingredient list and instructions for each dish. This allows users to easily plan healthy meals and reliably obtain the necessary ingredients.

[1117] "User device" refers to an electronic device such as a computer, smartphone, or tablet that a user uses to input information.

[1118] A "database" is a collection of electronic files that systematically stores information received from users and manages it so that it can be easily accessed and searched later.

[1119] A "generative AI model" is a component of artificial intelligence that uses machine learning algorithms to analyze user input and generate optimal menus.

[1120] "AI means" refers to systems or programs that use artificial intelligence technology to analyze data and perform specific tasks.

[1121] "Meal candidates" refer to multiple meal plans generated based on user input.

[1122] "Ingredient Delivery Service Provider" refers to a service provider that delivers the necessary ingredients according to the ingredients list based on the menu selected by the user.

[1123] This invention aims to provide users with healthy, balanced meals by reducing the hassle and stress of daily meal planning and ingredient purchasing. The system consists of a smartphone application, a cloud server, and a generative AI model. Each function of the system and its implementation are described below.

[1124] 1. Enter your user information

[1125] User device:

[1126] Users access the dedicated application using their smartphone or tablet and enter their past meal plan information, desired type of food, desired calorie intake, and disliked foods. Specifically, users enter their meal history from the past month, their preferred food category (e.g., Japanese food, Chinese food), their daily calorie intake (e.g., 1700 kcal / day), and allergy information (e.g., nuts, shrimp).

[1127] 2. Receipt and storage of data

[1128] server:

[1129] The server receives the information sent from the user's device and stores it in a database, thereby obtaining individual user information that can be used for subsequent data analysis.

[1130] 3. Menu generation

[1131] server:

[1132] The server retrieves the saved user information and analyzes it using a generative AI model. The generative AI model, which implements a machine learning algorithm (using TensorFlow or PyTorch, for example), generates multiple menu candidates that take into account past eating history, preferences, nutritional balance, and calorie intake. Examples of generated menu candidates include "rice balls and miso soup" for breakfast, "fried rice" for lunch, and "grilled salted mackerel" for dinner.

[1133] 4. Presentation and selection of menu options

[1134] server:

[1135] The five generated menu options are sent to the user's device.

[1136] User device:

[1137] The user reviews the proposed menu options and selects the most suitable one.

[1138] 5. Material list generation and automatic delivery linkage

[1139] server:

[1140] Based on the menu selected by the user, a one-month menu is confirmed and an ingredient list is generated according to the confirmed menu. The generated ingredient list is linked to a food delivery system on the cloud, and the necessary ingredients are automatically delivered to the user. The food delivery company periodically delivers the necessary ingredients to the user based on the generated ingredient list.

[1141] 6. Providing cooking instructions

[1142] server:

[1143] The server provides detailed recipes for each dish in the generated menu, with cooking instructions in text and video formats and links to external recipe sites.

[1144] User device:

[1145] Users check the provided list of ingredients and cooking instructions, and then actually cook the dish. For example, detailed instructions for making onigiri are displayed, such as "Step 1: Cook rice. Step 2: Shape the onigiri." Users can refer to videos and links to improve the quality of their cooking.

[1146] Examples and prompts

[1147] Examples:

[1148] The user inputs "I'm allergic to nuts and shrimp, and I like Japanese and Chinese food," and sets their calorie intake. The next month's menu is generated, consisting of "rice balls, miso soup, fried rice, and grilled salted mackerel," and the necessary ingredients are delivered by a food delivery company.

[1149] Example prompt sentence:

[1150] The user enters information about the past month's menu, favorite types of food, calorie intake, allergy information, etc. This information is used to create a balanced menu for the next month. Favorite types of food are "Japanese" and "Chinese," calorie intake is "1700 kcal / day," and allergy information is "nuts and shrimp." The generated menu should be "Breakfast: rice balls and miso soup, lunch: fried rice, dinner: grilled salted mackerel."

[1151] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1152] Step 1:

[1153] The user's device inputs past menu information, desired type of food, desired calorie intake, and disliked foods. The user then launches the application and inputs their meal history for the past month, their preferred food category (e.g., Japanese, Chinese), their daily calorie intake (e.g., 1700 kcal / day), and allergy information (e.g., nuts, shrimp). This information is temporarily stored on the user's device and sent for later processing.

[1154] Step 2:

[1155] The user device sends the input information to the server, which receives it and stores it in a database. This information includes the user's dietary history, preferences, allergy information, calorie intake, etc. The database is structured to allow efficient access and searching.

[1156] Step 3:

[1157] The server retrieves the stored user information from the database and analyzes it using a generative AI model. The server passes the retrieved data as input to the generative AI model (using TensorFlow and PyTorch). The model analyzes the user's past eating history and preferences, and generates optimal meal suggestions taking into account nutritional balance and calorie intake. Five types of meal suggestions are generated as output.

[1158] Step 4:

[1159] The server sends the five generated menu candidates to the user's device. The user's device receives this information and displays it on the screen. The user reviews the presented menu candidates and selects the most suitable one. The information on the selected menu is temporarily stored on the user's device and then sent back to the server.

[1160] Step 5:

[1161] The server determines a monthly menu based on the menu selected by the user. It then generates a list of ingredients required based on the determined menu. The ingredient list lists the ingredients required for each dish in the selected menu. The generated ingredient list is then sent to a food delivery system on the cloud.

[1162] Step 6:

[1163] The food delivery company prepares the necessary ingredients based on the ingredient list received from the server and creates a delivery schedule. The delivery company delivers the necessary ingredients periodically based on the user's address information. This ensures that the user can obtain the ingredients they need.

[1164] Step 7:

[1165] The server provides detailed instructions for each dish in the generated menu. The cooking instructions are prepared in text and video formats and displayed within the application. Links to external recipe websites are also displayed. Users can check this information and use it as a reference when actually cooking.

[1166] These are the specific processing steps of the system program that realizes the application example. At each step, it is clear what information is required as input and what data processing and calculations are performed to obtain the output.

[1167] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1168] This system aims to reduce the effort and stress involved in daily menu planning and ingredient purchasing for users, and to provide healthy, balanced meals. Furthermore, by combining it with an emotion engine, it can also provide personalized menu suggestions based on the user's emotions. Below, we will explain an embodiment of the present invention when combined with an emotion engine.

[1169] Entering user information

[1170] User device:

[1171] Users access a dedicated application using their own smartphone, computer, or other device and log in to the system. They enter menu information from the past month, the type of food they want (Japanese, Chinese, etc.), their desired calorie intake, and any foods they dislike (including allergens). For example, a user might enter "toast and salad" for breakfast, "spaghetti" for lunch, and "gratin" for dinner in the past. They might also select "Japanese, Chinese" as their preferences, specify their calorie intake as "1700 kcal / day," and enter "nuts, shrimp" as their disliked foods.

[1172] Receiving and storing data

[1173] server:

[1174] The server receives the information entered from the user's terminal and stores it in a database, which stores the user's personal information and preferences in the system for subsequent processing.

[1175] emotion recognition

[1176] User device:

[1177] Users input their feedback on the food, their daily physical condition, and their mood. The emotion engine also recognizes emotions from the user's facial expressions and tone of voice. This information is sent from the user's device to the server.

[1178] Menu generation

[1179] server:

[1180] The server retrieves the user's stored information from the database, as well as the user's emotional data recognized by the emotion engine. Based on this information, the system performs analysis using a machine learning algorithm. Based on this analysis, it generates five menu suggestions that take into account nutritional balance, calorie intake, and the user's emotions. For example, if the user is feeling stressed, it will suggest a menu using ingredients that have a relaxing effect.

[1181] Presentation and selection of menu options

[1182] server:

[1183] The server sends the five generated menu options to the user's terminal.

[1184] User device:

[1185] The user reviews the menu options and selects the most suitable one. For example, the user selects "Option 1."

[1186] Generate a materials list

[1187] server:

[1188] Receives menu information selected by the user and determines a monthly menu based on the selection. Generates a list of ingredients required based on the determined menu. For example, the list may include "rice, miso, mackerel, and vegetables."

[1189] Providing cooking instructions

[1190] server:

[1191] The server provides cooking instructions for each dish included in the confirmed menu. This includes cooking instructions in text format and cooking videos. It also displays links to major external cooking recipe sites. For example, the server provides text information such as "Step 1: Cook rice. Step 2: Shape the rice balls" for how to make rice balls, along with a video explaining how to cook grilled salted mackerel.

[1192] User device:

[1193] Users can check the provided ingredient list and cooking instructions, then actually cook the dish, and refer to videos and links to improve the quality of their cooking.

[1194] Accumulating emotional data and reflecting it in the next proposal

[1195] server:

[1196] The system accumulates user emotional data and reflects it in the next menu suggestion. Based on the user's past emotional data, it is possible to suggest more optimized menus. By continuously learning from this data, the system's accuracy improves.

[1197] In this way, the system can utilize the user's personal information and emotional data to efficiently plan meals and purchase ingredients, making it easy to prepare balanced, healthy meals.

[1198] The processing flow will be explained below.

[1199] Step 1:

[1200] User device:

[1201] The user logs in to the system. After logging in, a special input form is displayed. Using this form, the user enters menu information for the past month. Specifically, the user enters information such as "January 1st: Breakfast - toast and salad, lunch - spaghetti, dinner - gratin."

[1202] Step 2:

[1203] User device:

[1204] The user selects the type of cuisine they desire. For example, they select "Japanese, Chinese." They also input their desired daily calorie intake. For example, they input "1700 kcal / day." They also input foods they dislike or are allergic to. For example, they input "nuts, shrimp."

[1205] Step 3:

[1206] server:

[1207] It receives information entered from the user's device and stores it in a database, which allows the system to store information about the user's personal information and preferences for subsequent processing.

[1208] Step 4:

[1209] User device:

[1210] Users input their feedback, daily physical condition, and mood. For example, they can enter comments such as "I feel stressed today" or "I feel relaxed today." The emotion engine then analyzes the user's facial expressions and tone of voice to recognize their emotions. This information is sent from the user's device to the server.

[1211] Step 5:

[1212] server:

[1213] The system receives emotional data sent from the user's device and stores it in a database, allowing the user's emotional state to be accumulated and used for future suggestions.

[1214] Step 6:

[1215] server:

[1216] The system retrieves user information and emotional data stored in the database. Based on this information, it uses a machine learning algorithm to perform analysis. Based on this analysis, it generates five menu candidates that take into account nutritional balance, calorie intake, and the user's emotions. For example, if the user is feeling stressed, it will suggest a menu using ingredients that have a relaxing effect.

[1217] Step 7:

[1218] server:

[1219] The five generated menu options are sent to the user's device.

[1220] Step 8:

[1221] User device:

[1222] The user reviews the five menu options displayed. After reviewing the details of each option, the user can select the option that best suits them. For example, the user can select "Option 1."

[1223] Step 9:

[1224] server:

[1225] Receives menu information selected by the user and determines a monthly menu based on the selection. Generates a list of ingredients required based on the determined menu. For example, the list may include "rice, miso, mackerel, and vegetables."

[1226] Step 10:

[1227] server:

[1228] It prepares recipes for each dish based on the confirmed menu, including text-based cooking instructions and video-based cooking instructions, and also generates links to major external recipe sites.

[1229] Step 11:

[1230] User device:

[1231] Users are provided with a list of ingredients and instructions for each dish, and are guided through videos and links to create the dish, making it easy for users to prepare a balanced meal.

[1232] Step 12:

[1233] server:

[1234] The system accumulates user emotional data and reflects it in the next menu suggestion. Based on the user's past emotional data, it is possible to suggest more optimized menus. By continuously learning from this data, the system's accuracy improves.

[1235] Example 2

[1236] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1237] Conventional menu creation systems only consider the user's preferences and nutritional balance, and are unable to provide personalized suggestions that reflect the user's mood or physical condition at any given time. As a result, users spend a lot of time and effort trying to find meals that suit their mood and physical condition, which can cause mental stress. This issue is especially important in today's world, where healthy meal choices that take into account nutritional balance and calorie intake are required.

[1238] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1239] In this invention, the server includes: means for a user to input past menu information, desired types of dishes, desired calorie intake, and disliked foods; means for receiving the input information and saving it in a database; means for analyzing the user's facial expressions and voice tone using an emotion recognition engine to generate emotion data; generation AI model means equipped with a machine learning algorithm that analyzes the saved information and emotion data and generates multiple menu candidates taking into account nutritional balance, calorie intake, and the user's emotions; means for transmitting the generated menu candidates to a user terminal; means for the user to select from the generated menu candidates; means for generating an ingredient list based on the menu selected by the user; and means for providing the ingredient list and cooking instructions for each dish. This enables personalized menu suggestions that reflect the user's emotions and physical condition, allowing users to easily prepare healthy and balanced meals.

[1240] "User" refers to an individual who uses the system, and in particular, is an entity that inputs information to create menus and purchase ingredients.

[1241] "Emotion recognition engine" refers to software or hardware technology that analyzes a user's facial expressions and vocal tone to generate emotional data for the user.

[1242] "Machine learning algorithms" refers to a set of computational methods and models for analyzing data and learning patterns, which are used here to generate meal suggestions.

[1243] A "generative AI model" refers to the entire system that uses machine learning algorithms to analyze user information and emotional data and generate optimal menu suggestions.

[1244] A "database" refers to a collection of information that accumulates information and emotional data input by users and is used in subsequent processing, as well as a system that manages this information.

[1245] "User terminal" refers to a device such as a smartphone or computer that allows a user to input information, display and select menu options, and check cooking instructions.

[1246] "Menu candidates" refer to multiple menu plans that the generative AI model suggests by analyzing the user's information and emotional data.

[1247] "Ingredient list" refers to a list of ingredients, seasonings, etc. required based on the menu options selected by the user.

[1248] "How to cook" refers to information that explains the steps to cook a dish based on a list of ingredients, and is provided in text or video format.

[1249] This system is intended to streamline user menu planning and ingredient purchasing, and to provide personalized meal suggestions based on emotions. Specific embodiments of this system will be described below.

[1250] Entering user information

[1251] User device:

[1252] Users open a dedicated application and log in to the system by entering the necessary authentication information on the login screen. After logging in, users enter information about breakfast, lunch, and dinner from the past month, the type of food they prefer (Japanese, Chinese, etc.), their desired daily calorie intake, and any food allergies they dislike. For example, they can enter information such as "toast and salad," "spaghetti," "gratin," "Japanese, Chinese," "1700 kcal / day," and "nuts, shrimp."

[1253] Receiving and storing data

[1254] server:

[1255] The server receives various information sent from the user's device. The received information is stored in a database and accumulated as profile information for each user. This database uses a relational database such as MySQL or PostgreSQL.

[1256] emotion recognition

[1257] User device:

[1258] Users input their daily physical condition, mood, and feedback on cooking into the application. The app also uses a camera and microphone with a built-in emotion recognition engine to measure the user's facial expressions and voice tone to generate emotion data. For example, if a user says, "I'm tired today," the emotion recognition engine analyzes this and recognizes it as "feeling tired."

[1259] server:

[1260] The server receives the emotion data sent from the user's device and stores it in a database.

[1261] Menu generation

[1262] server:

[1263] The server retrieves user information and emotional data from the database in parallel and executes a Python script to process them. The script performs analysis using machine learning algorithms (TensorFlow and PyTorch). As a result, it generates five types of meal suggestions that take into account nutritional balance, calorie intake, and emotional data. For example, if the user is determined to be "feeling tired," the suggestions will include meals using ingredients with a relaxing effect (herbal tea or warm soup).

[1264] Presentation and selection of menu options

[1265] server:

[1266] The server sends the five generated menu options to the user's device via API and reflects them in real time.

[1267] User device:

[1268] The user checks five menu options presented on the dedicated application screen and selects the most suitable option. For example, the user selects "Option 1."

[1269] Generate a materials list

[1270] server:

[1271] The server receives the user's selections and finalizes a monthly menu plan. Based on the finalized menu, it generates a list of ingredients needed, such as rice, miso, mackerel, and vegetables.

[1272] Providing cooking instructions

[1273] server:

[1274] The server retrieves cooking instructions for each dish in the confirmed menu from the database and sends them to the user's device. Cooking instructions are provided in text format and in the form of videos linked to external sites such as YouTube. For example, "How to make onigiri" includes text information such as "Step 1: Cook rice. Step 2: Shape the onigiri." as well as a video link explaining how to cook grilled salted mackerel.

[1275] User device:

[1276] Users can then review the provided ingredient list and cooking instructions, and use videos and links to help guide them through the cooking process.

[1277] Accumulating emotional data and reflecting it in the next proposal

[1278] server:

[1279] The server continuously accumulates user emotional data and uses it to propose the next menu. This data is analyzed by a machine learning algorithm, contributing to improving the accuracy of the entire system. Based on the user's past emotional data, it is possible to propose even more optimized menus.

[1280] Example prompt sentence:

[1281] "I'm tired today. Can you suggest a relaxing meal?"

[1282] In this way, the system utilizes the user's individual information and emotional data to create efficient and balanced menus and purchase ingredients.

[1283] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1284] Step 1:

[1285] Users open a dedicated application and enter the required authentication information on the login screen to log in to the system.

[1286] Input: Authentication information (user ID, password)

[1287] Output: Login success status

[1288] Specific operation: After successfully logging in, the user enters their dietary information for the past month, the type of food they want to eat, their daily calorie goal, and any food they dislike. For example, they can enter "toast and salad," "spaghetti," "gratin," "Japanese and Chinese food," "1700 kcal / day," and "nuts and shrimp."

[1289] Step 2:

[1290] The server receives parameters such as dietary information, preferences, and allergy information sent from the user terminal.

[1291] Input: User-entered information (past menus, desired type of food, calorie goal, disliked ingredients)

[1292] Output: User information stored in the database

[1293] Specific operation: The received information is stored in a database and accumulated as basic data for the system to make suggestions. This database is built using MySQL or PostgreSQL.

[1294] Step 3:

[1295] Users input their daily physical condition and emotional feedback through the application, and an emotion recognition engine analyzes the user's facial expressions and voice tone to generate emotional data.

[1296] Input: User's physical condition feedback, voice data, face image

[1297] Output: Emotion data (e.g., tiredness, stress, relaxation)

[1298] What it does: The user types "I'm tired today," and the system uses the camera and microphone to capture and analyze facial expressions and voice tone for sentiment analysis.

[1299] Step 4:

[1300] The server stores the emotion data received from the user's device in a database.

[1301] Input: Emotion data

[1302] Output: Emotion data stored in a database

[1303] Specific operation: The server receives emotion data in real time and stores it in a database along with existing user data.

[1304] Step 5:

[1305] The server retrieves stored user information and emotional data from the database and analyzes it using machine learning algorithms.

[1306] Input: User information, emotion data

[1307] Output: Menu suggestions (e.g., menus using ingredients with a relaxing effect)

[1308] How it works: Using TensorFlow and PyTorch, the system analyzes nutritional balance, calories, and emotional data to generate five different menu options. For example, it suggests relaxing herbal tea or warm soup to a user who is feeling tired.

[1309] Step 6:

[1310] The server sends the generated menu candidates to the user's terminal via an API.

[1311] Input: Generated menu candidates

[1312] Output: Menu suggestions displayed on the user's device

[1313] Specific operation: Data is sent from the server to the user's device via API, and menu suggestions are displayed in real time on the application screen.

[1314] Step 7:

[1315] The user reviews five possible menu options and selects the one they most desire.

[1316] Input: Menu candidate

[1317] Output: Selected menu

[1318] Specific operation: The user refers to the menu options displayed within the app and performs an operation such as selecting "Option 1."

[1319] Step 8:

[1320] The server receives the menu options selected by the user and finalizes the menu plan for one month.

[1321] Input: Selected menu

[1322] Output: Finalized menu and ingredients list

[1323] Specific operation: The server generates a list of necessary ingredients based on the selected menu, listing "rice, miso, mackerel, vegetables," etc.

[1324] Step 9:

[1325] Based on the confirmed menu, the server retrieves the recipes for each dish from the database and sends them to the user's terminal.

[1326] Input: Confirmed menu

[1327] Output: Recipe for each dish (text format, video format)

[1328] Specific operation: The server summarizes the cooking steps in text format and sends cooking steps such as "Step 1: Cook rice. Step 2: Shape the rice balls," as well as a video link explaining how to cook grilled salted mackerel.

[1329] Step 10:

[1330] Users check the provided ingredient list and cooking instructions and then actually create the dish.

[1331] Input: Ingredient list, recipe

[1332] Output: Finished dish

[1333] Specific actions: The user refers to the app and proceeds with cooking based on the provided instructions and videos. For example, watching the provided video, the user makes grilled salted mackerel.

[1334] Step 11:

[1335] The server continuously accumulates the user's emotional data and reflects it in the next menu suggestion.

[1336] Input: Daily emotion data

[1337] Output: Improve the accuracy of the next suggestion using accumulated emotion data

[1338] How it works: The server analyzes user sentiment data collected daily and uses it to make the next menu suggestion. Through continuous learning, the system can make more advanced suggestions.

[1339] Through the above processing steps, users can efficiently create personalized menus based on their emotions and physical condition, and easily prepare healthy and balanced meals.

[1340] (Application example 2)

[1341] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1342] The problem that this invention aims to solve is to provide healthy and balanced meals by reducing the effort and stress of daily menu planning and ingredient purchasing for users, and to provide a system that takes into account the user's emotional state, proposes personalized menus, and enables instant delivery orders.

[1343] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for the user to input past menu information, desired dish types, desired calorie intake, and disliked foods; means for receiving the input information and saving it in a database; AI means for analyzing the saved information and generating multiple menu candidates taking nutritional balance and calorie intake into consideration; means for transmitting the generated menu candidates to a user terminal; means for the user to select from the generated menu candidates; means for generating an ingredient list based on the menu selected by the user; means for providing the ingredient list and cooking instructions for each dish; emotion recognition means for recognizing the user's emotions and personalizing the menu based on the emotions; and ordering means for placing a delivery order based on the emotion-based recommended menu. This enables the provision of personalized, healthy, and balanced meals that take the user's emotions into consideration. Furthermore, the server allows for instant ordering through a delivery service, enhancing user convenience.

[1344] "User information" refers to data such as menu information, desired types of food, desired calorie intake, and disliked foods that the user has previously entered.

[1345] A "database" is a storage device that stores information received from users and uses it for subsequent processing.

[1346] "Nutritional balance" refers to the appropriate proportions and combinations of nutrients contained in food, and is important for maintaining a healthy diet.

[1347] "Calorie intake" refers to the amount of energy a user takes in per day, and is the energy value of a meal.

[1348] "Meal suggestions" are meal options suggested based on the user's preferences and health status.

[1349] "Emotion recognition means" refers to devices or technologies that analyze and recognize a user's emotional state from facial expressions, vocal tone, etc.

[1350] "Personalization" means optimizing experiences and services based on a user's individual preferences and feelings.

[1351] "Delivery Order" means the ordering process for the delivery of the meal selected by the User to the specified location.

[1352] "AI means" is a technology that uses machine learning and data analysis to process user information and generate optimal menus.

[1353] The "Ingredient List" is a list of ingredients required to prepare the selected menu.

[1354] "How-to" refers to the steps or methods for preparing a meal, and is an instruction provided in text or video format.

[1355] As part of the system, this invention reduces the effort required for users to plan daily meals and purchase ingredients, provides healthy and balanced meals, and even suggests personalized meals through emotion recognition. This system is implemented mainly through a user device such as a smartphone and a server.

[1356] Entering user information

[1357] Users access a dedicated application using their own smartphone or other device and log in to the system. Here, they input past menu information, the type of food they want (Japanese, Chinese, etc.), their desired calorie intake, and any food dislikes (including allergens). For example, a user might input "toast and salad" for breakfast, "spaghetti" for lunch, and "gratin" for dinner, select "Japanese, Chinese" as their preference, and input their calorie intake as "1700 kcal / day" and "nuts, shrimp" as their food dislikes. This information is sent from the user's device to the server and stored in a database.

[1358] Receiving and storing data

[1359] The server receives the information entered from the user's device and stores it in a database, which stores the user's personal information and preferences in the system for use in subsequent processing.

[1360] emotion recognition

[1361] Users input emotional information such as feedback on the cooking experience, daily physical condition, and mood. The app also incorporates a function to recognize emotions from the user's facial expressions and voice tone using emotion recognition techniques, such as Microsoft Azure Face API and OpenCV. This information is sent from the user's device to the server.

[1362] Menu generation

[1363] The server retrieves user information stored in the database and user emotion data recognized by the emotion recognition means. Based on this data, it analyzes it using a machine learning algorithm (for example, TensorFlow or Scikit-learn) to generate multiple menu candidates that take into account nutritional balance, calorie intake, and the user's emotions. For example, if the user is feeling stressed, it will suggest a menu using ingredients that have a relaxing effect.

[1364] Presentation and selection of menu options

[1365] The server sends the generated menu candidates to the user's terminal. The user reviews the presented menu candidates and selects the most suitable one. For example, the user selects "Candidate 1." This selection information is sent to the server.

[1366] Generate a materials list

[1367] The server receives the menu information selected by the user and generates a list of ingredients based on the selected menu, such as "rice, miso, mackerel, and vegetables."

[1368] Providing cooking instructions

[1369] The server provides cooking instructions for each dish according to the confirmed menu. This includes cooking instructions in text format and cooking videos. It also displays links to external cooking recipe sites. For example, the server provides text information such as "Step 1: Cook rice. Step 2: Shape the rice balls" for how to make rice balls, as well as a video explaining how to cook grilled salted mackerel.

[1370] Accumulating emotional data and reflecting it in the next proposal

[1371] The server accumulates the user's emotional data and reflects it in the next menu suggestion. Based on the user's past emotional data, it is possible to suggest a more optimized menu. By continuously learning from this data, the accuracy of the system improves.

[1372] Delivery Order

[1373] The system has a means for placing a delivery order based on the menu information selected by the user. For example, the system performs a procedure that seamlessly integrates the experience to order ingredients for batter from a delivery service based on the menu selected by the user.

[1374] Prompt Sentence Examples

[1375] Recommend and order the best meal based on the user's food preferences and emotional data. If the user is feeling stressed, suggest a relaxing dish. Create it based on the following information:

[1376] Past meals: Breakfast: "Toast", Lunch: "Salad", Dinner: "Curry"

[1377] Preferences: "Japanese food", "Chinese food"

[1378] Disliked foods: "Nuts", "Shrimp"

[1379] Today's mood: "I'm stressed"

[1380] Convert the suggested menu into a delivery order.

[1381] This allows us to provide personalized, healthy, and balanced meals that take into account the user's emotions, and also allows for instant ordering through delivery services, increasing user convenience.

[1382] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1383] Step 1:

[1384] The user inputs past menu information, desired dish types, desired calorie intake, and disliked foods into the user's device. This input information includes the user's past breakfast choices of "toast and salad," lunch choices of "spaghetti," and dinner choices of "gratin." The input information is sent to the server.

[1385] Step 2:

[1386] The server stores the information received from the user's device in a database, accumulating data on the user's preferences and nutritional intake for subsequent analysis. The input data is stored as is, and undergoes format conversion and field alignment.

[1387] Step 3:

[1388] The user terminal inputs emotional information such as daily physical condition and mood. An emotion recognition means is also used to recognize emotions from facial expressions and voice tones. The emotional information (e.g., "stress") is sent to the server.

[1389] Step 4:

[1390] The server uses a machine learning algorithm to analyze the emotion data acquired by the emotion recognition means and past menu information. This analysis generates multiple menu candidates that take into account nutritional balance and calorie intake. The input is emotion data and user preference data, and the output is the generated menu candidates.

[1391] Step 5:

[1392] The server sends the generated menu options to the user's device. For example, five menu options are displayed, each with its nutritional value and calorie count. The user can visually confirm these options.

[1393] Step 6:

[1394] The user terminal selects the most suitable menu from the presented menu candidates. This selection information is sent back to the server. The input is the selected menu, and the output is detailed information about the selected menu.

[1395] Step 7:

[1396] The server generates a list of ingredients based on the selected menu information. The input is the selected menu information, and the output is a list of ingredients (e.g., "rice, miso, mackerel, vegetables"). This list is checked against a database to define the required ingredients in detail.

[1397] Step 8:

[1398] The server provides recipes for each dish based on a confirmed menu. This includes text-based cooking instructions and video-based cooking instructions. The input is the menu information, and the output is links to the cooking instructions and videos.

[1399] Step 9:

[1400] The user device displays the provided ingredient list and cooking instructions, and then actually cooks the dish, allowing the user to cook with a high level of perfection.

[1401] Step 10:

[1402] The server accumulates the user's emotional data and menu selection history and reflects this in the next menu suggestion. The input is the user's emotional history and selection history, and the output is data that contributes to improving the accuracy of the next suggestion.

[1403] Step 11:

[1404] The user terminal places a delivery order based on the selected menu. This information is sent to the server, which then executes the delivery service. The input is the user's order information, and the output is order confirmation and delivery execution information.

[1405] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1406] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1407] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1408] [Fourth embodiment]

[1409] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1410] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1411] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1412] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1413] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1414] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1415] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1416] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1417] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1418] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1419] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1420] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1421] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1422] The purpose of this system is to reduce the effort and stress involved in daily menu planning and ingredient purchasing for users, and to provide healthy and balanced meals. The following describes the embodiments of the present invention.

[1423] Entering user information

[1424] User device:

[1425] Users access a dedicated application using their smartphone, computer, or other device and log in to the system. They input menu information from the past month, the type of food they prefer (Japanese, Chinese, etc.), their desired calorie intake, and any foods they dislike (including allergens). For example, a user might input "toast and salad" for breakfast, "spaghetti" for lunch, and "gratin" for dinner in the past. They might also select "Japanese, Chinese" as their preferences, specify their calorie intake as "1700 kcal / day," and input "nuts, shrimp" as foods they dislike.

[1426] Receiving and storing data

[1427] server:

[1428] The server receives the information sent from the user's device and stores it in a database, thereby retaining the individual user information needed for subsequent processing and analysis.

[1429] Menu generation

[1430] server:

[1431] The server retrieves the stored user information from the database. It then analyzes this information using a machine learning algorithm to generate the ideal menu for the next month, taking into account nutritional balance and calorie intake. Specifically, the AI ​​analyzes the user's past eating history and health status, and automatically generates five menu candidates. For example, one of the generated candidates might be "Breakfast: Rice balls and miso soup, Lunch: Fried rice, Dinner: Grilled salted mackerel."

[1432] Presentation and selection of menu options

[1433] server:

[1434] The server sends the five generated menu options to the user's terminal.

[1435] User device:

[1436] The user reviews the menu options and selects the most suitable one. For example, the user selects "Option 1."

[1437] Generate a materials list

[1438] server:

[1439] The server determines a monthly menu based on the menu selected by the user. A list of ingredients is generated based on the determined menu. For example, an ingredient list based on the determined menu may include "rice, miso, mackerel, and vegetables."

[1440] Providing cooking instructions

[1441] server:

[1442] The server provides instructions for each dish included in the confirmed menu. Detailed cooking instructions are prepared in the form of text and video, and links to major external recipe websites are also displayed. For example, when it comes to how to make rice balls, text information such as "Step 1: Cook rice. Step 2: Shape the rice balls" is provided, along with a video explaining how to cook grilled salted mackerel.

[1443] User device:

[1444] Users can check the provided ingredient list and cooking instructions, then actually cook the dish, and refer to videos and links to improve the quality of their cooking.

[1445] In this way, users can efficiently plan and shop for ingredients, and easily prepare balanced, healthy meals.

[1446] The processing flow will be explained below.

[1447] Step 1:

[1448] User device:

[1449] The user logs in to the system. After logging in, a special input form is displayed. Using this form, the user enters menu information for the past month. Specifically, the user enters information such as "January 1st: Breakfast - toast and salad, lunch - spaghetti, dinner - gratin."

[1450] Step 2:

[1451] User device:

[1452] The user selects the type of cuisine they desire. For example, they select "Japanese, Chinese." They also input their desired daily calorie intake. For example, they input "1700 kcal / day." They also input foods they dislike or are allergic to. For example, they input "nuts, shrimp."

[1453] Step 3:

[1454] server:

[1455] It receives information entered from the user's device and stores it in a database, which allows the system to store information about the user's personal information and preferences for subsequent processing.

[1456] Step 4:

[1457] server:

[1458] The system obtains the user's menu information and preferences for the past month from the database. Based on the obtained information, it uses a machine learning algorithm to perform an analysis. Based on this analysis, it generates five menu candidates that take into account nutritional balance and calorie intake. For example, candidate 1 would be "Breakfast: rice balls and miso soup, lunch: fried rice, dinner: grilled salted mackerel."

[1459] Step 5:

[1460] server:

[1461] The five generated menu options are sent to the user's device.

[1462] Step 6:

[1463] User device:

[1464] The user reviews the five menu options displayed. After reviewing the details of each option, the user can select the option that best suits them. For example, the user can select "Option 1."

[1465] Step 7:

[1466] server:

[1467] Receives menu information selected by the user and determines a monthly menu based on the selection. Based on the determined menu, generates a list of required ingredients. For example, the list may include "rice, miso, mackerel, and vegetables."

[1468] Step 8:

[1469] server:

[1470] It prepares recipes for each dish based on the confirmed menu, including text-based cooking instructions and video-based cooking instructions, and also generates links to major external recipe sites.

[1471] Step 9:

[1472] User device:

[1473] Users are provided with a list of ingredients and instructions for each dish, and are guided through videos and links to create the dish, making it easy for users to prepare a balanced meal.

[1474] This series of steps allows the system to significantly reduce the effort required for users to create menus and purchase ingredients, and supports healthy eating habits.

[1475] Example 1

[1476] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1477] In today's busy lifestyles, planning daily meals and purchasing ingredients can be a time-consuming and stressful process. It is also difficult and time-consuming to create a plan that takes nutritional balance into account. As a result, achieving a healthy and balanced diet can be challenging.

[1478] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1479] In this invention, the server includes a means for allowing a user to input past meal plan information, favorite types of dishes, desired energy intake, and ingredients to avoid, a means for receiving the input information and storing it in a data structure, and a generative AI model means for analyzing the stored information and generating multiple meal plan candidates taking into account nutritional balance and energy intake, thereby enabling a user to efficiently create menus, purchase necessary ingredients, and easily prepare balanced, healthy meals.

[1480] "User" refers to a person who uses the system to create menus and purchase ingredients.

[1481] "Means of input" refers to the mechanism by which users use a terminal to provide the system with information such as past meal planning information, preferred types of food, desired energy intake, and ingredients to avoid.

[1482] "Data Structure" refers to the arrangement of information within the system for organizing and temporarily or permanently storing information received from users.

[1483] "Generative AI model" refers to an artificial intelligence algorithm that generates meal plan suggestions that take into account nutritional balance and energy intake based on stored user information.

[1484] "Display Device" means a device for visually presenting generated meal plan suggestions and other information to a user, including, for example, a smartphone screen or a computer monitor.

[1485] "Ingredients List" refers to the list of ingredients needed to prepare a meal based on the meal plan selected by the User.

[1486] "Cooking instructions" refers to content that provides instructions and methods for preparing a dish using ingredients in text and video format.

[1487] "Energy intake" refers to the amount of calories a user needs per day.

[1488] The system of this invention allows users to efficiently plan menus and purchase ingredients, and easily prepare balanced, healthy meals. This system is implemented primarily using a user terminal and a server.

[1489] Entering user information

[1490] User terminal

[1491] Users log in to a dedicated application using a device such as a smartphone or computer. After logging in, users are prompted to enter the following information:

[1492] Menu information for the past month

[1493] For example: "Breakfast: Toast and salad", "Lunch: Spaghetti", "Dinner: Gratin"

[1494] Desired type of cuisine (Japanese, Chinese, etc.)

[1495] Example: "Japanese food, Chinese food"

[1496] Desired calorie intake

[1497] Example: "1700 kcal / day"

[1498] Foods you dislike (including allergens)

[1499] For example: "Nuts, shrimp."

[1500] Receiving and storing data

[1501] server

[1502] The information entered by the user is sent from the user terminal, and the server receives this information, which is then stored in a data structure for further processing.

[1503] Menu generation

[1504] server

[1505] The server retrieves the stored user information from the data structure and analyzes it using the generative AI model. Specifically, it performs the following steps:

[1506] Obtain the user's dietary history and health information.

[1507] Data is input into a generative AI model to generate multiple meal plan options that take into account nutritional balance and energy intake.

[1508] Example: "Breakfast: rice balls and miso soup, lunch: fried rice, dinner: grilled salted mackerel"

[1509] Presentation and selection of menu options

[1510] server

[1511] The server sends the generated meal plan candidates to the user terminal.

[1512] User terminal

[1513] The user checks the displayed menu options and selects the most suitable one. The selected information is sent to the server and used for the next process.

[1514] Generate a materials list

[1515] server

[1516] The server determines the menu for one month based on the selected menu and generates a list of ingredients, which is then sent to the user's device.

[1517] Example: "Rice, miso, mackerel, vegetables"

[1518] Providing cooking instructions

[1519] server

[1520] Based on the confirmed menu, the server provides cooking instructions for each dish, including text and video formats, as well as links to major external recipe sites.

[1521] Examples: "How to make onigiri: Step 1: Cook the rice. Step 2: Shape the onigiri" or "Video explaining how to cook grilled salted mackerel"

[1522] User terminal

[1523] Users can check the provided ingredient list and cooking instructions, then actually cook the dish, and refer to videos and links to improve the quality of their cooking.

[1524] In this way, the system helps users efficiently plan, purchase, and prepare balanced, healthy meals.

[1525] "Please enter the user's menu information for the past month. Example: Breakfast: Toast and salad, Lunch: Spaghetti, Dinner: Gratin"

[1526] "Please select the type of cuisine you would like. Example: Japanese, Chinese"

[1527] "Enter your desired daily calorie intake. Example: 1700 kcal / day"

[1528] "Please enter any foods you dislike or are allergic to. Example: nuts, shrimp."

[1529] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1530] Step 1: Enter your user information

[1531] User terminal

[1532] Users log in to a dedicated application using their smartphone or computer, and then enter the following information when prompted:

[1533] Menu information for the past month (e.g., "Breakfast: Toast and Salad," "Lunch: Spaghetti," "Dinner: Gratin")

[1534] Desired type of cuisine (e.g., "Japanese, Chinese")

[1535] Desired calorie intake (e.g., "1700 kcal / day")

[1536] Foods you dislike (e.g., "nuts, shrimp")

[1537] The input data sent to the server includes this user information.

[1538] Step 2: Receiving and storing data

[1539] server

[1540] The server receives the information sent from the user device. Specifically, when the input data reaches the server's API endpoint, the server performs data validation and format conversion, then executes a query and stores it in the database in the appropriate format.

[1541] Input: User's diet history, favorite dishes, desired calorie intake, ingredients to avoid

[1542] Output: Saved user information is recorded in the database

[1543] Step 3: Create a menu

[1544] server

[1545] The server retrieves stored user information from the database. Based on the retrieved data, it uses a generative AI model to generate multiple meal plan candidates that take into account nutritional balance and calorie intake. Specifically, the server performs the following steps:

[1546] Query the database to retrieve the user's dietary history and related information

[1547] Input the acquired data into the generative AI model and start the analysis

[1548] As a result of the analysis, five types of meal plan candidates are generated.

[1549] Example: "Breakfast: rice balls and miso soup, lunch: fried rice, dinner: grilled salted mackerel"

[1550] Input: User information stored in the database

[1551] Output: 5 meal plan candidates generated

[1552] Step 4: Presentation and selection of menu options

[1553] server

[1554] The generated meal plan candidates are sent to the user's device.

[1555] User terminal

[1556] The user reviews the proposed meal plans and selects the most suitable one. Specifically, the application displays a list of meal plan candidates, and when the user taps to select one, the selection information is sent to the server.

[1557] Input: 5 meal plan options

[1558] Output: The meal plan selected by the user

[1559] Step 5: Generate a materials list

[1560] server

[1561] The server determines a monthly menu based on the menu selected by the user. It then generates a list of ingredients required for this determined menu. Specifically, the server analyzes the menu information and lists the ingredients required for each dish. The list is then saved in a database and sent to the user's device.

[1562] Example: "Rice, miso, mackerel, vegetables"

[1563] Input: Menu information selected by the user

[1564] Output: Generated materials list

[1565] Step 6: Provide cooking instructions

[1566] server

[1567] The server provides cooking instructions for each dish based on the confirmed menu. This is provided in both text and video format, and also displays links to major cooking recipe sites. Specifically, it retrieves the relevant cooking instructions from a recipe database, generates video links and text data, and sends them to the user's device.

[1568] Examples: "How to make onigiri: Step 1: Cook the rice. Step 2: Shape the onigiri" or "Video explaining how to cook grilled salted mackerel"

[1569] User terminal

[1570] Users can check the provided ingredient list and cooking instructions, then actually create the dish, especially by referring to videos and links to improve the quality of the dish.

[1571] Input: Confirmed menu information

[1572] Output: Cooking instructions in text and video format

[1573] In this way, the system helps users efficiently plan meals, purchase the necessary ingredients, and prepare balanced, healthy meals.

[1574] (Application example 1)

[1575] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1576] The objective of this invention is to provide healthy and balanced meals by reducing the effort and stress involved in daily menu planning and ingredient purchasing for users. It also aims to provide a system that allows users to easily select menus, efficiently obtain the necessary ingredients, and prepare meals. In particular, it aims to realize a system that, by linking with food delivery services, saves users the trouble of purchasing ingredients on the spot.

[1577] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1578] In this invention, the server includes: means for a user to input past menu information, desired types of dishes, desired calorie intake, and disliked foods; means for receiving the input information and saving it in a database; AI means using a generative AI model that analyzes the saved information and generates multiple menu candidates taking into account nutritional balance and calorie intake; means for sending the generated menu candidates to a user terminal; means for the user to select from the generated menu candidates; means for generating a list of ingredients based on the menu selected by the user and linking it to a food delivery company; and means for providing the ingredient list and instructions for each dish. This allows users to easily plan healthy meals and reliably obtain the necessary ingredients.

[1579] "User device" refers to an electronic device such as a computer, smartphone, or tablet that a user uses to input information.

[1580] A "database" is a collection of electronic files that systematically stores information received from users and manages it so that it can be easily accessed and searched later.

[1581] A "generative AI model" is a component of artificial intelligence that uses machine learning algorithms to analyze user input and generate optimal menus.

[1582] "AI means" refers to systems or programs that use artificial intelligence technology to analyze data and perform specific tasks.

[1583] "Meal candidates" refer to multiple meal plans generated based on user input.

[1584] "Ingredient Delivery Service Provider" refers to a service provider that delivers the necessary ingredients according to the ingredients list based on the menu selected by the user.

[1585] This invention aims to provide users with healthy, balanced meals by reducing the hassle and stress of daily meal planning and ingredient purchasing. The system consists of a smartphone application, a cloud server, and a generative AI model. Each function of the system and its implementation are described below.

[1586] 1. Enter your user information

[1587] User device:

[1588] Users access the dedicated application using their smartphone or tablet and enter their past meal plan information, desired type of food, desired calorie intake, and disliked foods. Specifically, users enter their meal history from the past month, their preferred food category (e.g., Japanese food, Chinese food), their daily calorie intake (e.g., 1700 kcal / day), and allergy information (e.g., nuts, shrimp).

[1589] 2. Receipt and storage of data

[1590] server:

[1591] The server receives the information sent from the user's device and stores it in a database, thereby obtaining individual user information that can be used for subsequent data analysis.

[1592] 3. Menu generation

[1593] server:

[1594] The server retrieves the saved user information and analyzes it using a generative AI model. The generative AI model, which implements a machine learning algorithm (using TensorFlow or PyTorch, for example), generates multiple menu candidates that take into account past eating history, preferences, nutritional balance, and calorie intake. Examples of generated menu candidates include "rice balls and miso soup" for breakfast, "fried rice" for lunch, and "grilled salted mackerel" for dinner.

[1595] 4. Presentation and selection of menu options

[1596] server:

[1597] The five generated menu options are sent to the user's device.

[1598] User device:

[1599] The user reviews the proposed menu options and selects the most suitable one.

[1600] 5. Material list generation and automatic delivery linkage

[1601] server:

[1602] Based on the menu selected by the user, a one-month menu is confirmed and an ingredient list is generated according to the confirmed menu. The generated ingredient list is linked to a food delivery system on the cloud, and the necessary ingredients are automatically delivered to the user. The food delivery company periodically delivers the necessary ingredients to the user based on the generated ingredient list.

[1603] 6. Providing cooking instructions

[1604] server:

[1605] The server provides detailed recipes for each dish in the generated menu, with cooking instructions in text and video formats and links to external recipe sites.

[1606] User device:

[1607] Users check the provided list of ingredients and cooking instructions, and then actually cook the dish. For example, detailed instructions for making onigiri are displayed, such as "Step 1: Cook rice. Step 2: Shape the onigiri." Users can refer to videos and links to improve the quality of their cooking.

[1608] Examples and prompts

[1609] Examples:

[1610] The user inputs "I'm allergic to nuts and shrimp, and I like Japanese and Chinese food," and sets their calorie intake. The next month's menu is generated, consisting of "rice balls, miso soup, fried rice, and grilled salted mackerel," and the necessary ingredients are delivered by a food delivery company.

[1611] Example prompt sentence:

[1612] The user enters information about the past month's menu, favorite types of food, calorie intake, allergy information, etc. This information is used to create a balanced menu for the next month. Favorite types of food are "Japanese" and "Chinese," calorie intake is "1700 kcal / day," and allergy information is "nuts and shrimp." The generated menu should be "Breakfast: rice balls and miso soup, lunch: fried rice, dinner: grilled salted mackerel."

[1613] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1614] Step 1:

[1615] The user's device inputs past menu information, desired type of food, desired calorie intake, and disliked foods. The user then launches the application and inputs their meal history for the past month, their preferred food category (e.g., Japanese, Chinese), their daily calorie intake (e.g., 1700 kcal / day), and allergy information (e.g., nuts, shrimp). This information is temporarily stored on the user's device and sent for later processing.

[1616] Step 2:

[1617] The user device sends the input information to the server, which receives it and stores it in a database. This information includes the user's dietary history, preferences, allergy information, calorie intake, etc. The database is structured to allow efficient access and searching.

[1618] Step 3:

[1619] The server retrieves the stored user information from the database and analyzes it using a generative AI model. The server passes the retrieved data as input to the generative AI model (using TensorFlow and PyTorch). The model analyzes the user's past eating history and preferences, and generates optimal meal suggestions taking into account nutritional balance and calorie intake. Five types of meal suggestions are generated as output.

[1620] Step 4:

[1621] The server sends the five generated menu candidates to the user's device. The user's device receives this information and displays it on the screen. The user reviews the presented menu candidates and selects the most suitable one. The information on the selected menu is temporarily stored on the user's device and then sent back to the server.

[1622] Step 5:

[1623] The server determines a monthly menu based on the menu selected by the user. It then generates a list of ingredients required based on the determined menu. The ingredient list lists the ingredients required for each dish in the selected menu. The generated ingredient list is then sent to a food delivery system on the cloud.

[1624] Step 6:

[1625] The food delivery company prepares the necessary ingredients based on the ingredient list received from the server and creates a delivery schedule. The delivery company delivers the necessary ingredients periodically based on the user's address information. This ensures that the user can obtain the ingredients they need.

[1626] Step 7:

[1627] The server provides detailed instructions for each dish in the generated menu. The cooking instructions are prepared in text and video formats and displayed within the application. Links to external recipe websites are also displayed. Users can check this information and use it as a reference when actually cooking.

[1628] These are the specific processing steps of the system program that realizes the application example. At each step, it is clear what information is required as input and what data processing and calculations are performed to obtain the output.

[1629] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1630] This system aims to reduce the effort and stress involved in daily menu planning and ingredient purchasing for users, and to provide healthy, balanced meals. Furthermore, by combining it with an emotion engine, it can also provide personalized menu suggestions based on the user's emotions. Below, we will explain an embodiment of the present invention when combined with an emotion engine.

[1631] Entering user information

[1632] User device:

[1633] Users access a dedicated application using their own smartphone, computer, or other device and log in to the system. They enter menu information from the past month, the type of food they want (Japanese, Chinese, etc.), their desired calorie intake, and any foods they dislike (including allergens). For example, a user might enter "toast and salad" for breakfast, "spaghetti" for lunch, and "gratin" for dinner in the past. They might also select "Japanese, Chinese" as their preferences, specify their calorie intake as "1700 kcal / day," and enter "nuts, shrimp" as their disliked foods.

[1634] Receiving and storing data

[1635] server:

[1636] The server receives the information entered from the user's terminal and stores it in a database, which stores the user's personal information and preferences in the system for subsequent processing.

[1637] emotion recognition

[1638] User device:

[1639] Users input their feedback on the food, their daily physical condition, and their mood. The emotion engine also recognizes emotions from the user's facial expressions and tone of voice. This information is sent from the user's device to the server.

[1640] Menu generation

[1641] server:

[1642] The server retrieves the user's stored information from the database, as well as the user's emotional data recognized by the emotion engine. Based on this information, the system performs analysis using a machine learning algorithm. Based on this analysis, it generates five menu suggestions that take into account nutritional balance, calorie intake, and the user's emotions. For example, if the user is feeling stressed, it will suggest a menu using ingredients that have a relaxing effect.

[1643] Presentation and selection of menu options

[1644] server:

[1645] The server sends the five generated menu options to the user's terminal.

[1646] User device:

[1647] The user reviews the menu options and selects the most suitable one. For example, the user selects "Option 1."

[1648] Generate a materials list

[1649] server:

[1650] Receives menu information selected by the user and determines a monthly menu based on the selection. Generates a list of ingredients required based on the determined menu. For example, the list may include "rice, miso, mackerel, and vegetables."

[1651] Providing cooking instructions

[1652] server:

[1653] The server provides cooking instructions for each dish included in the confirmed menu. This includes cooking instructions in text format and cooking videos. It also displays links to major external cooking recipe sites. For example, the server provides text information such as "Step 1: Cook rice. Step 2: Shape the rice balls" for how to make rice balls, along with a video explaining how to cook grilled salted mackerel.

[1654] User device:

[1655] Users can check the provided ingredient list and cooking instructions, then actually cook the dish, and refer to videos and links to improve the quality of their cooking.

[1656] Accumulating emotional data and reflecting it in the next proposal

[1657] server:

[1658] The system accumulates user emotional data and reflects it in the next menu suggestion. Based on the user's past emotional data, it is possible to suggest more optimized menus. By continuously learning from this data, the system's accuracy improves.

[1659] In this way, the system can utilize the user's personal information and emotional data to efficiently plan meals and purchase ingredients, making it easy to prepare balanced, healthy meals.

[1660] The processing flow will be explained below.

[1661] Step 1:

[1662] User device:

[1663] The user logs in to the system. After logging in, a special input form is displayed. Using this form, the user enters menu information for the past month. Specifically, the user enters information such as "January 1st: Breakfast - toast and salad, lunch - spaghetti, dinner - gratin."

[1664] Step 2:

[1665] User device:

[1666] The user selects the type of cuisine they desire. For example, they select "Japanese, Chinese." They also input their desired daily calorie intake. For example, they input "1700 kcal / day." They also input foods they dislike or are allergic to. For example, they input "nuts, shrimp."

[1667] Step 3:

[1668] server:

[1669] It receives information entered from the user's device and stores it in a database, which allows the system to store information about the user's personal information and preferences for subsequent processing.

[1670] Step 4:

[1671] User device:

[1672] Users input their feedback, daily physical condition, and mood. For example, they can enter comments such as "I feel stressed today" or "I feel relaxed today." The emotion engine then analyzes the user's facial expressions and tone of voice to recognize their emotions. This information is sent from the user's device to the server.

[1673] Step 5:

[1674] server:

[1675] The system receives emotional data sent from the user's device and stores it in a database, allowing the user's emotional state to be accumulated and used for future suggestions.

[1676] Step 6:

[1677] server:

[1678] The system retrieves user information and emotional data stored in the database. Based on this information, it uses a machine learning algorithm to perform analysis. Based on this analysis, it generates five menu candidates that take into account nutritional balance, calorie intake, and the user's emotions. For example, if the user is feeling stressed, it will suggest a menu using ingredients that have a relaxing effect.

[1679] Step 7:

[1680] server:

[1681] The five generated menu options are sent to the user's device.

[1682] Step 8:

[1683] User device:

[1684] The user reviews the five menu options displayed. After reviewing the details of each option, the user can select the option that best suits them. For example, the user can select "Option 1."

[1685] Step 9:

[1686] server:

[1687] Receives menu information selected by the user and determines a monthly menu based on the selection. Generates a list of ingredients required based on the determined menu. For example, the list may include "rice, miso, mackerel, and vegetables."

[1688] Step 10:

[1689] server:

[1690] It prepares recipes for each dish based on the confirmed menu, including text-based cooking instructions and video-based cooking instructions, and also generates links to major external recipe sites.

[1691] Step 11:

[1692] User device:

[1693] Users are provided with a list of ingredients and instructions for each dish, and are guided through videos and links to create the dish, making it easy for users to prepare a balanced meal.

[1694] Step 12:

[1695] server:

[1696] The system accumulates user emotional data and reflects it in the next menu suggestion. Based on the user's past emotional data, it is possible to suggest more optimized menus. By continuously learning from this data, the system's accuracy improves.

[1697] Example 2

[1698] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1699] Conventional menu creation systems only consider the user's preferences and nutritional balance, and are unable to provide personalized suggestions that reflect the user's mood or physical condition at any given time. As a result, users spend a lot of time and effort trying to find meals that suit their mood and physical condition, which can cause mental stress. This issue is especially important in today's world, where healthy meal choices that take into account nutritional balance and calorie intake are required.

[1700] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1701] In this invention, the server includes: means for a user to input past menu information, desired types of dishes, desired calorie intake, and disliked foods; means for receiving the input information and saving it in a database; means for analyzing the user's facial expressions and voice tone using an emotion recognition engine to generate emotion data; generation AI model means equipped with a machine learning algorithm that analyzes the saved information and emotion data and generates multiple menu candidates taking into account nutritional balance, calorie intake, and the user's emotions; means for transmitting the generated menu candidates to a user terminal; means for the user to select from the generated menu candidates; means for generating an ingredient list based on the menu selected by the user; and means for providing the ingredient list and cooking instructions for each dish. This enables personalized menu suggestions that reflect the user's emotions and physical condition, allowing users to easily prepare healthy and balanced meals.

[1702] "User" refers to an individual who uses the system, and in particular, is an entity that inputs information to create menus and purchase ingredients.

[1703] "Emotion recognition engine" refers to software or hardware technology that analyzes a user's facial expressions and vocal tone to generate emotional data for the user.

[1704] "Machine learning algorithms" refers to a set of computational methods and models for analyzing data and learning patterns, which are used here to generate meal suggestions.

[1705] A "generative AI model" refers to the entire system that uses machine learning algorithms to analyze user information and emotional data and generate optimal menu suggestions.

[1706] A "database" refers to a collection of information that accumulates information and emotional data input by users and is used in subsequent processing, as well as a system that manages this information.

[1707] "User terminal" refers to a device such as a smartphone or computer that allows a user to input information, display and select menu options, and check cooking instructions.

[1708] "Menu candidates" refer to multiple menu plans that the generative AI model suggests by analyzing the user's information and emotional data.

[1709] "Ingredient list" refers to a list of ingredients, seasonings, etc. required based on the menu options selected by the user.

[1710] "How to cook" refers to information that explains the steps to cook a dish based on a list of ingredients, and is provided in text or video format.

[1711] This system is intended to streamline user menu planning and ingredient purchasing, and to provide personalized meal suggestions based on emotions. Specific embodiments of this system will be described below.

[1712] Entering user information

[1713] User device:

[1714] Users open a dedicated application and log in to the system by entering the necessary authentication information on the login screen. After logging in, users enter information about breakfast, lunch, and dinner from the past month, the type of food they prefer (Japanese, Chinese, etc.), their desired daily calorie intake, and any food allergies they dislike. For example, they can enter information such as "toast and salad," "spaghetti," "gratin," "Japanese, Chinese," "1700 kcal / day," and "nuts, shrimp."

[1715] Receiving and storing data

[1716] server:

[1717] The server receives various information sent from the user's device. The received information is stored in a database and accumulated as profile information for each user. This database uses a relational database such as MySQL or PostgreSQL.

[1718] emotion recognition

[1719] User device:

[1720] Users input their daily physical condition, mood, and feedback on cooking into the application. The app also uses a camera and microphone with a built-in emotion recognition engine to measure the user's facial expressions and voice tone to generate emotion data. For example, if a user says, "I'm tired today," the emotion recognition engine analyzes this and recognizes it as "feeling tired."

[1721] server:

[1722] The server receives the emotion data sent from the user's device and stores it in a database.

[1723] Menu generation

[1724] server:

[1725] The server retrieves user information and emotional data from the database in parallel and executes a Python script to process them. The script performs analysis using machine learning algorithms (TensorFlow and PyTorch). As a result, it generates five types of meal suggestions that take into account nutritional balance, calorie intake, and emotional data. For example, if the user is determined to be "feeling tired," the suggestions will include meals using ingredients with a relaxing effect (herbal tea or warm soup).

[1726] Presentation and selection of menu options

[1727] server:

[1728] The server sends the five generated menu options to the user's device via API and reflects them in real time.

[1729] User device:

[1730] The user checks five menu options presented on the dedicated application screen and selects the most suitable option. For example, the user selects "Option 1."

[1731] Generate a materials list

[1732] server:

[1733] The server receives the user's selections and finalizes a monthly menu plan. Based on the finalized menu, it generates a list of ingredients needed, such as rice, miso, mackerel, and vegetables.

[1734] Providing cooking instructions

[1735] server:

[1736] The server retrieves cooking instructions for each dish in the confirmed menu from the database and sends them to the user's device. Cooking instructions are provided in text format and in the form of videos linked to external sites such as YouTube. For example, "How to make onigiri" includes text information such as "Step 1: Cook rice. Step 2: Shape the onigiri." as well as a video link explaining how to cook grilled salted mackerel.

[1737] User device:

[1738] Users can then review the provided ingredient list and cooking instructions, and use videos and links to help guide them through the cooking process.

[1739] Accumulating emotional data and reflecting it in the next proposal

[1740] server:

[1741] The server continuously accumulates user emotional data and uses it to propose the next menu. This data is analyzed by a machine learning algorithm, contributing to improving the accuracy of the entire system. Based on the user's past emotional data, it is possible to propose even more optimized menus.

[1742] Example prompt sentence:

[1743] "I'm tired today. Can you suggest a relaxing meal?"

[1744] In this way, the system utilizes the user's individual information and emotional data to create efficient and balanced menus and purchase ingredients.

[1745] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1746] Step 1:

[1747] Users open a dedicated application and enter the required authentication information on the login screen to log in to the system.

[1748] Input: Authentication information (user ID, password)

[1749] Output: Login success status

[1750] Specific operation: After successfully logging in, the user enters their dietary information for the past month, the type of food they want to eat, their daily calorie goal, and any food they dislike. For example, they can enter "toast and salad," "spaghetti," "gratin," "Japanese and Chinese food," "1700 kcal / day," and "nuts and shrimp."

[1751] Step 2:

[1752] The server receives parameters such as dietary information, preferences, and allergy information sent from the user terminal.

[1753] Input: User-entered information (past menus, desired type of food, calorie goal, disliked ingredients)

[1754] Output: User information stored in the database

[1755] Specific operation: The received information is stored in a database and accumulated as basic data for the system to make suggestions. This database is built using MySQL or PostgreSQL.

[1756] Step 3:

[1757] Users input their daily physical condition and emotional feedback through the application, and an emotion recognition engine analyzes the user's facial expressions and voice tone to generate emotional data.

[1758] Input: User's physical condition feedback, voice data, face image

[1759] Output: Emotion data (e.g., tiredness, stress, relaxation)

[1760] What it does: The user types "I'm tired today," and the system uses the camera and microphone to capture and analyze facial expressions and voice tone for sentiment analysis.

[1761] Step 4:

[1762] The server stores the emotion data received from the user's device in a database.

[1763] Input: Emotion data

[1764] Output: Emotion data stored in a database

[1765] Specific operation: The server receives emotion data in real time and stores it in a database along with existing user data.

[1766] Step 5:

[1767] The server retrieves stored user information and emotional data from the database and analyzes it using machine learning algorithms.

[1768] Input: User information, emotion data

[1769] Output: Menu suggestions (e.g., menus using ingredients with a relaxing effect)

[1770] How it works: Using TensorFlow and PyTorch, the system analyzes nutritional balance, calories, and emotional data to generate five different menu options. For example, it suggests relaxing herbal tea or warm soup to a user who is feeling tired.

[1771] Step 6:

[1772] The server sends the generated menu candidates to the user's terminal via an API.

[1773] Input: Generated menu candidates

[1774] Output: Menu suggestions displayed on the user's device

[1775] Specific operation: Data is sent from the server to the user's device via API, and menu suggestions are displayed in real time on the application screen.

[1776] Step 7:

[1777] The user reviews five possible menu options and selects the one they most desire.

[1778] Input: Menu candidate

[1779] Output: Selected menu

[1780] Specific operation: The user refers to the menu options displayed within the app and performs an operation such as selecting "Option 1."

[1781] Step 8:

[1782] The server receives the menu options selected by the user and finalizes the menu plan for one month.

[1783] Input: Selected menu

[1784] Output: Finalized menu and ingredients list

[1785] Specific operation: The server generates a list of necessary ingredients based on the selected menu, listing "rice, miso, mackerel, vegetables," etc.

[1786] Step 9:

[1787] Based on the confirmed menu, the server retrieves the recipes for each dish from the database and sends them to the user's terminal.

[1788] Input: Confirmed menu

[1789] Output: Recipe for each dish (text format, video format)

[1790] Specific operation: The server summarizes the cooking steps in text format and sends cooking steps such as "Step 1: Cook rice. Step 2: Shape the rice balls," as well as a video link explaining how to cook grilled salted mackerel.

[1791] Step 10:

[1792] Users check the provided ingredient list and cooking instructions and then actually create the dish.

[1793] Input: Ingredient list, recipe

[1794] Output: Finished dish

[1795] Specific actions: The user refers to the app and proceeds with cooking based on the provided instructions and videos. For example, watching the provided video, the user makes grilled salted mackerel.

[1796] Step 11:

[1797] The server continuously accumulates the user's emotional data and reflects it in the next menu suggestion.

[1798] Input: Daily emotion data

[1799] Output: Improve the accuracy of the next suggestion using accumulated emotion data

[1800] How it works: The server analyzes user sentiment data collected daily and uses it to make the next menu suggestion. Through continuous learning, the system can make more advanced suggestions.

[1801] Through the above processing steps, users can efficiently create personalized menus based on their emotions and physical condition, and easily prepare healthy and balanced meals.

[1802] (Application example 2)

[1803] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1804] The problem that this invention aims to solve is to provide healthy and balanced meals by reducing the effort and stress of daily menu planning and ingredient purchasing for users, and to provide a system that takes into account the user's emotional state, proposes personalized menus, and enables instant delivery orders.

[1805] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for the user to input past menu information, desired dish types, desired calorie intake, and disliked foods; means for receiving the input information and saving it in a database; AI means for analyzing the saved information and generating multiple menu candidates taking nutritional balance and calorie intake into consideration; means for transmitting the generated menu candidates to a user terminal; means for the user to select from the generated menu candidates; means for generating an ingredient list based on the menu selected by the user; means for providing the ingredient list and cooking instructions for each dish; emotion recognition means for recognizing the user's emotions and personalizing the menu based on the emotions; and ordering means for placing a delivery order based on the emotion-based recommended menu. This enables the provision of personalized, healthy, and balanced meals that take the user's emotions into consideration. Furthermore, the server allows for instant ordering through a delivery service, enhancing user convenience.

[1806] "User information" refers to data such as menu information, desired types of food, desired calorie intake, and disliked foods that the user has previously entered.

[1807] A "database" is a storage device that stores information received from users and uses it for subsequent processing.

[1808] "Nutritional balance" refers to the appropriate proportions and combinations of nutrients contained in food, and is important for maintaining a healthy diet.

[1809] "Calorie intake" refers to the amount of energy a user takes in per day, and is the energy value of a meal.

[1810] "Meal suggestions" are meal options suggested based on the user's preferences and health status.

[1811] "Emotion recognition means" refers to devices or technologies that analyze and recognize a user's emotional state from facial expressions, vocal tone, etc.

[1812] "Personalization" means optimizing experiences and services based on a user's individual preferences and feelings.

[1813] "Delivery Order" means the ordering process for the delivery of the meal selected by the User to the specified location.

[1814] "AI means" is a technology that uses machine learning and data analysis to process user information and generate optimal menus.

[1815] The "Ingredient List" is a list of ingredients required to prepare the selected menu.

[1816] "How-to" refers to the steps or methods for preparing a meal, and is an instruction provided in text or video format.

[1817] As part of the system, this invention reduces the effort required for users to plan daily meals and purchase ingredients, provides healthy and balanced meals, and even suggests personalized meals through emotion recognition. This system is implemented mainly through a user device such as a smartphone and a server.

[1818] Entering user information

[1819] Users access a dedicated application using their own smartphone or other device and log in to the system. Here, they input past menu information, the type of food they want (Japanese, Chinese, etc.), their desired calorie intake, and any food dislikes (including allergens). For example, a user might input "toast and salad" for breakfast, "spaghetti" for lunch, and "gratin" for dinner, select "Japanese, Chinese" as their preference, and input their calorie intake as "1700 kcal / day" and "nuts, shrimp" as their food dislikes. This information is sent from the user's device to the server and stored in a database.

[1820] Receiving and storing data

[1821] The server receives the information entered from the user's device and stores it in a database, which stores the user's personal information and preferences in the system for use in subsequent processing.

[1822] emotion recognition

[1823] Users input emotional information such as feedback on the cooking experience, daily physical condition, and mood. The app also incorporates a function to recognize emotions from the user's facial expressions and voice tone using emotion recognition techniques, such as Microsoft Azure Face API and OpenCV. This information is sent from the user's device to the server.

[1824] Menu generation

[1825] The server retrieves user information stored in the database and user emotion data recognized by the emotion recognition means. Based on this data, it analyzes it using a machine learning algorithm (for example, TensorFlow or Scikit-learn) to generate multiple menu candidates that take into account nutritional balance, calorie intake, and the user's emotions. For example, if the user is feeling stressed, it will suggest a menu using ingredients that have a relaxing effect.

[1826] Presentation and selection of menu options

[1827] The server sends the generated menu candidates to the user's terminal. The user reviews the presented menu candidates and selects the most suitable one. For example, the user selects "Candidate 1." This selection information is sent to the server.

[1828] Generate a materials list

[1829] The server receives the menu information selected by the user and generates a list of ingredients based on the selected menu, such as "rice, miso, mackerel, and vegetables."

[1830] Providing cooking instructions

[1831] The server provides cooking instructions for each dish according to the confirmed menu. This includes cooking instructions in text format and cooking videos. It also displays links to external cooking recipe sites. For example, the server provides text information such as "Step 1: Cook rice. Step 2: Shape the rice balls" for how to make rice balls, as well as a video explaining how to cook grilled salted mackerel.

[1832] Accumulating emotional data and reflecting it in the next proposal

[1833] The server accumulates the user's emotional data and reflects it in the next menu suggestion. Based on the user's past emotional data, it is possible to suggest a more optimized menu. By continuously learning from this data, the accuracy of the system improves.

[1834] Delivery Order

[1835] The system has a means for placing a delivery order based on the menu information selected by the user. For example, the system performs a procedure that seamlessly integrates the experience to order ingredients for batter from a delivery service based on the menu selected by the user.

[1836] Prompt Sentence Examples

[1837] Recommend and order the best meal based on the user's food preferences and emotional data. If the user is feeling stressed, suggest a relaxing dish. Create it based on the following information:

[1838] Past meals: Breakfast: "Toast", Lunch: "Salad", Dinner: "Curry"

[1839] Preferences: "Japanese food", "Chinese food"

[1840] Disliked foods: "Nuts", "Shrimp"

[1841] Today's mood: "I'm stressed"

[1842] Convert the suggested menu into a delivery order.

[1843] This allows us to provide personalized, healthy, and balanced meals that take into account the user's emotions, and also allows for instant ordering through delivery services, increasing user convenience.

[1844] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1845] Step 1:

[1846] The user inputs past menu information, desired dish types, desired calorie intake, and disliked foods into the user's device. This input information includes the user's past breakfast choices of "toast and salad," lunch choices of "spaghetti," and dinner choices of "gratin." The input information is sent to the server.

[1847] Step 2:

[1848] The server stores the information received from the user's device in a database, accumulating data on the user's preferences and nutritional intake for subsequent analysis. The input data is stored as is, and undergoes format conversion and field alignment.

[1849] Step 3:

[1850] The user terminal inputs emotional information such as daily physical condition and mood. An emotion recognition means is also used to recognize emotions from facial expressions and voice tones. The emotional information (e.g., "stress") is sent to the server.

[1851] Step 4:

[1852] The server uses a machine learning algorithm to analyze the emotion data acquired by the emotion recognition means and past menu information. This analysis generates multiple menu candidates that take into account nutritional balance and calorie intake. The input is emotion data and user preference data, and the output is the generated menu candidates.

[1853] Step 5:

[1854] The server sends the generated menu options to the user's device. For example, five menu options are displayed, each with its nutritional value and calorie count. The user can visually confirm these options.

[1855] Step 6:

[1856] The user terminal selects the most suitable menu from the presented menu candidates. This selection information is sent back to the server. The input is the selected menu, and the output is detailed information about the selected menu.

[1857] Step 7:

[1858] The server generates a list of ingredients based on the selected menu information. The input is the selected menu information, and the output is a list of ingredients (e.g., "rice, miso, mackerel, vegetables"). This list is checked against a database to define the required ingredients in detail.

[1859] Step 8:

[1860] The server provides recipes for each dish based on a confirmed menu. This includes text-based cooking instructions and video-based cooking instructions. The input is the menu information, and the output is links to the cooking instructions and videos.

[1861] Step 9:

[1862] The user device displays the provided ingredient list and cooking instructions, and then actually cooks the dish, allowing the user to cook with a high level of perfection.

[1863] Step 10:

[1864] The server accumulates the user's emotional data and menu selection history and reflects this in the next menu suggestion. The input is the user's emotional history and selection history, and the output is data that contributes to improving the accuracy of the next suggestion.

[1865] Step 11:

[1866] The user terminal places a delivery order based on the selected menu. This information is sent to the server, which then executes the delivery service. The input is the user's order information, and the output is order confirmation and delivery execution information.

[1867] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1868] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1869] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1870] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1871] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1872] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1873] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1874] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1875] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1876] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1877] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1878] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1879] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1880] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1881] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1882] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1883] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1884] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1885] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1886] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1887] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1888] The following is further disclosed regarding the above embodiment.

[1889] (Claim 1)

[1890] A way for users to input past menu information, desired type of food, desired calorie intake, and disliked foods,

[1891] means for receiving the input information and storing it in a database;

[1892] An AI means for analyzing the stored information and generating multiple menu candidates taking into consideration nutritional balance and calorie intake;

[1893] A means for transmitting the generated menu candidates to a user terminal;

[1894] A means for allowing a user to select from the generated menu candidates;

[1895] means for generating a list of ingredients required based on a user-selected menu;

[1896] means for providing the ingredient list and recipes for each dish;

[1897] A system including:

[1898] (Claim 2)

[1899] 2. The system according to claim 1, further comprising means for categorizing the generated menu candidates into specific cuisine types such as Japanese cuisine and Chinese cuisine and providing them.

[1900] (Claim 3)

[1901] 10. The system of claim 1, further comprising means for providing instructions for preparing each of the dishes in the form of text and video.

[1902] "Example 1"

[1903] (Claim 1)

[1904] A means for users to input past meal planning information, preferred food types, desired energy intake, and ingredients to avoid;

[1905] means for receiving and storing said input information in a data structure;

[1906] a generating AI model means for analyzing the stored information and generating a plurality of meal plan candidates taking into consideration nutritional balance and energy intake;

[1907] means for transmitting the generated candidate meal plans to a display device;

[1908] a means for a user to select from among the generated meal plan candidates;

[1909] means for generating a list of ingredients based on a user-selected meal plan;

[1910] means for providing the list of ingredients and cooking instructions for each dish;

[1911] A system including:

[1912] (Claim 2)

[1913] The system according to claim 1, further comprising means for categorizing the generated meal plan candidates into specific cuisine types, such as Japanese cuisine or Chinese cuisine, and providing the categorization results.

[1914] (Claim 3)

[1915] 10. The system of claim 1, further comprising means for providing cooking instructions for each of the dishes in the form of text and video.

[1916] "Application Example 1"

[1917] (Claim 1)

[1918] A way for users to input past menu information, desired type of food, desired calorie intake, and disliked foods,

[1919] means for receiving the input information and storing it in a database;

[1920] An AI means using a generation AI model that analyzes the stored information and generates multiple menu candidates taking into account nutritional balance and calorie intake;

[1921] A means for transmitting the generated menu candidates to a user terminal;

[1922] A means for allowing a user to select from the generated menu candidates;

[1923] A means for generating a list of ingredients based on the menu selected by the user and linking it to a food delivery company;

[1924] means for providing the ingredient list and recipes for each dish;

[1925] A system including:

[1926] (Claim 2)

[1927] 2. The system according to claim 1, further comprising means for categorizing the generated menu candidates into specific types of dishes and providing them.

[1928] (Claim 3)

[1929] 10. The system of claim 1, further comprising means for providing recipes for each dish in the form of text and video and displaying them on a user terminal.

[1930] "Example 2: Combining Emotion Engines"

[1931] (Claim 1)

[1932] A way for users to input past menu information, desired type of food, desired calorie intake, and disliked foods,

[1933] means for receiving the input information and storing it in a database;

[1934] means for analyzing a user's facial expression and voice tone using an emotion recognition engine to generate emotion data;

[1935] A generation AI model means having a machine learning algorithm that analyzes the stored information and emotion data and generates multiple menu candidates taking into account nutritional balance, calorie intake, and the user's emotion;

[1936] A means for transmitting the generated menu candidates to a user terminal;

[1937] A means for allowing a user to select from the generated menu candidates;

[1938] means for generating a list of ingredients required based on a user-selected menu;

[1939] means for providing the ingredient list and recipes for each dish;

[1940] A system including:

[1941] (Claim 2)

[1942] 2. The system according to claim 1, further comprising means for categorizing the generated menu candidates into specific types of dishes and providing them.

[1943] (Claim 3)

[1944] 10. The system of claim 1, further comprising means for providing instructions for preparing each of the dishes in the form of text and video.

[1945] "Application example 2 when combining emotion engines"

[1946] (Claim 1)

[1947] A way for users to input past menu information, desired type of food, desired calorie intake, and disliked foods,

[1948] means for receiving the input information and storing it in a database;

[1949] An AI means for analyzing the stored information and generating multiple menu candidates taking into consideration nutritional balance and calorie intake;

[1950] A means for transmitting the generated menu candidates to a user terminal;

[1951] A means for allowing a user to select from the generated menu candidates;

[1952] means for generating a list of ingredients required based on a user-selected menu;

[1953] means for providing the ingredient list and recipes for each dish;

[1954] emotion recognition means for recognizing an emotion of a user and personalizing a menu based on the emotion;

[1955] an ordering means for placing a delivery order based on the emotion-based recommended menu;

[1956] A system including:

[1957] (Claim 2)

[1958] 2. The system according to claim 1, further comprising means for categorizing the generated menu candidates into specific cuisine types such as Japanese cuisine and Chinese cuisine and providing them.

[1959] (Claim 3)

[1960] 10. The system of claim 1, further comprising means for providing instructions for preparing each of the dishes in the form of text and video. [Explanation of symbols]

[1961] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A way for users to input past menu information, desired type of food, desired calorie intake, and disliked foods, means for receiving the input information and storing it in a database; An AI means for analyzing the stored information and generating multiple menu candidates taking into consideration nutritional balance and calorie intake; A means for transmitting the generated menu candidates to a user terminal; A means for allowing a user to select from the generated menu candidates; means for generating a list of ingredients required based on a user-selected menu; means for providing the ingredient list and recipes for each dish; A system including:

2. The system according to claim 1 , further comprising means for categorizing the generated menu candidates into specific cuisine types such as Japanese cuisine and Chinese cuisine and providing them.

3. The system of claim 1 further comprising means for providing instructions for preparing each of the dishes in the form of text and video.

Images