system

Abstract

A system is provided.SOLUTION: A system comprising: means for inputting and storing personal information including health information; means for inputting and storing food stock information in a home; means for generating a meal menu using the stored personal information and food information; means for displaying a necessary ingredient list based on the generated meal menu; means for ordering ingredients via an API of an online supermarket; means for converting order contents into a QR code and distributing the QR code; and means for storing purchased ingredient information and providing a cooking method.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The effort required to plan daily menus, the complexity of preparing meals that are suited to each individual's health, and the burden of managing ingredients are common problems in many households. Selecting a balanced menu that takes into account each individual's health information and using all ingredients without waste is particularly challenging. The purpose of the present invention is to solve these problems. [Means for solving the problem]

[0005] The present invention provides a means for inputting and storing personal information including health information, a means for inputting and storing information on food ingredients in the home, and a means for generating a meal menu using the stored personal information and food ingredient information and displaying a list of ingredients required based on the generated meal menu.

[0006] Furthermore, it includes a means for ordering ingredients via the online supermarket's API and a means for converting the order details into a QR code and distributing it. The system also provides a means for saving information on purchased ingredients and providing cooking instructions, as well as a means for recording information on leftover ingredients and reflecting it in the next menu. This system provides comprehensive support from selecting meal menus to providing cooking instructions and reducing food waste, making daily meal preparation and health management more efficient.

[0007] "Health information" refers to information relating to an individual's health condition, and specifically includes allergies, calorie intake goals, disease information, and the like.

[0008] "Personal information" refers to information about individual users, including age, gender, health information, and the like.

[0009] "Ingredient inventory information" is information about the types and amounts of ingredients currently available in the home.

[0010] A "meal menu" is a list showing the contents of meals for a specific period of time, including the names of specific dishes and their composition.

[0011] The "ingredient list" is a list of ingredients needed to create a meal menu, including the specific types of ingredients and their amounts.

[0012] An "online supermarket" is an online retail store where you can purchase food and daily necessities via the Internet.

[0013] "API" stands for Application Programming Interface, a standardized means of connecting different software.

[0014] A "QR code" is a type of two-dimensional barcode that visually represents information, and can be scanned to read text information, URLs, etc.

[0015] "Cooking methods" include steps and recipes for cooking using specified ingredients, as well as links to cooking videos.

[0016] A "cooking app" is software that provides cooking instructions and recipes, and is provided in a form that is accessible to users. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] As a form for implementing the present invention, we will specifically explain a series of systems that generate meal menus based on health information and ingredient inventory information, procure ingredients through collaboration with an online supermarket, and provide cooking methods.

[0039] User registration and initial settings

[0040] When a user uses the system for the first time, they first enter their profile information (age, gender, allergy information, calorie intake goal, etc.) into the device. The device sends the entered information to the server, which stores it. Next, the user enters information about food ingredients in their home (e.g., rice, chicken, vegetables, etc.) into the device, which also sends this information to the server and stores it.

[0041] Meal Menu Generation

[0042] The server retrieves the stored personal information and food inventory information and uses the AI ​​model to generate a meal menu for several days. For example, it generates a menu of "chicken and tomato soup, salad, and brown rice" based on a calorie intake target of 1500 kcal. The generated menu is sent from the server to the user's device.

[0043] Generate and order materials lists

[0044] Based on the received menu, the terminal displays a list of required ingredients (e.g., "300g chicken, 2 tomatoes, salad vegetables, 2 cups brown rice") to the user. The user can check this list and edit it as necessary (e.g., change the amount, add, or delete). The confirmed ingredient list is sent as an order from the terminal via the online supermarket's API. The online supermarket receives the order and sets the delivery date and time and collection method.

[0045] QR Code Generation and Distribution

[0046] Once the order details are confirmed, the server converts the information into a QR code and sends it to the user's device. The user can then use the QR code received on their device to confirm receipt at the online supermarket.

[0047] Save purchased ingredients information and provide cooking instructions

[0048] The user inputs the delivered ingredients into the device and sends that information to the server. The server saves the purchased ingredient information and, based on the saved data, retrieves specific cooking instructions and video links via the cooking app's API. This information is sent from the server to the user's device, and the user can cook while referring to the cooking instructions displayed on the device.

[0049] Managing leftover ingredients and creating your next recipe

[0050] The server analyzes and records leftover ingredient information based on the user's purchase and cooking history. This information is reflected in the next menu generation and creates recipes to minimize waste. For example, it might suggest a new salad recipe using unused tomatoes.

[0051] Specific examples

[0052] When a female user in her 30s uses the system for the first time, she first enters her profile (e.g., female in her 30s, no allergies, calorie intake goal 1500 kcal) and her household food inventory (e.g., rice, chicken, tomatoes) into the device. Once this information is saved on the server, the AI ​​generates a menu of "chicken and tomato soup, salad, brown rice" based on that information and sends it to the user.

[0053] Once the user confirms and edits the ingredient list and submits the order to the online supermarket, the order is converted into a QR code and sent to them. After that, the user re-enters the received ingredient information, and a link to a cooking video with detailed cooking instructions is displayed on the device. The system aims to simplify daily meal preparation and promote health maintenance and efficient ingredient management.

[0054] The present invention reduces the burden of meal planning and food ingredient management on the user, enabling them to maintain a healthy diet.

[0055] The processing flow will be explained below.

[0056] Step 1:

[0057] The user enters profile information (age, gender, allergy information, calorie intake goal, etc.) into the terminal.

[0058] Step 2:

[0059] The terminal transmits the entered profile information to the server.

[0060] Step 3:

[0061] The server stores the received profile information in a database.

[0062] Step 4:

[0063] The user inputs information about food ingredients in the home into the terminal.

[0064] Step 5:

[0065] The terminal transmits the input food stock information to the server.

[0066] Step 6:

[0067] The server stores the received food stock information in a database.

[0068] Step 7:

[0069] The server uses an AI model to generate a meal menu for several days based on the saved profile information and food inventory information.

[0070] Step 8:

[0071] The server transmits the generated meal menu to the user terminal.

[0072] Step 9:

[0073] The terminal displays the meal menu received from the server to the user.

[0074] Step 10:

[0075] The user reviews the meal menu and edits the ingredient list as needed.

[0076] Step 11:

[0077] The terminal sends the confirmed list of ingredients to the online supermarket's API.

[0078] Step 12:

[0079] The online supermarket processes the order based on the received ingredient list and sets the delivery date and time and collection method.

[0080] Step 13:

[0081] The server receives the order details from the online supermarket and converts them into a QR code.

[0082] Step 14:

[0083] The server sends the generated QR code to the user's terminal.

[0084] Step 15:

[0085] The user receives the QR code on their device and uses it to receive the ingredients.

[0086] Step 16:

[0087] The user inputs the delivered food information into the terminal.

[0088] Step 17:

[0089] The terminal transmits information about the delivered ingredients to the server.

[0090] Step 18:

[0091] The server stores the received purchased ingredient information in a database.

[0092] Step 19:

[0093] The server connects to the cooking app's API based on the stored ingredient information to obtain specific cooking instructions and video links.

[0094] Step 20:

[0095] The server sends the acquired cooking instructions and video links to the user's terminal.

[0096] Step 21:

[0097] The user cooks the food while referring to the cooking instructions displayed on the terminal.

[0098] Step 22:

[0099] The server analyzes and records information about leftover ingredients based on the purchase history and cooking history sent by the user.

[0100] Step 23:

[0101] The server will take the remaining ingredient information into consideration when generating a new menu next time.

[0102] This series of processing steps streamlines meal preparation and health management for users.

[0103] Example 1

[0104] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0105] Conventional meal planning systems require users to plan their own meal menus, source the ingredients, and search for cooking methods, which is time-consuming. They also tend to waste ingredients and make efficient food management difficult. There is a need for a system that can solve these problems and allow users to easily maintain healthy meals.

[0106] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0107] In this invention, the server includes means for inputting and saving user information including health information, means for inputting and saving household food inventory information, means for generating a meal menu using a generative AI model based on the saved user information and food inventory information, means for displaying a list of necessary ingredients based on the generated meal menu, means for ordering ingredients via an interface of an online sales service, means for coding and distributing the order details, means for saving information on purchased ingredients and providing cooking instructions, and means for recording information on leftover ingredients and reflecting it in generating the next menu. This allows users to easily plan healthy meals and efficiently procure and manage the necessary ingredients.

[0108] "Health information" refers to health-related data such as a user's age, gender, allergy information, and calorie intake goal.

[0109] "User Information" means general data about individuals using the system, including health information.

[0110] "Food inventory information" is data on the types and quantities of food currently held in the household.

[0111] "Storage means" refers to the methods and techniques used to record and retain input information in a database.

[0112] A "generative AI model" is an algorithm or system that uses artificial intelligence technology to generate specific information from input data.

[0113] A "meal menu" is a list of specific meals that a user plans to consume.

[0114] An "ingredient list" is a list of ingredients and quantities needed to prepare a particular meal.

[0115] An "interface for online retail services" is a means of connection to an online retailer's system for ordering products.

[0116] "Encoding" refers to converting data into a format such as a barcode or QR code.

[0117] "Purchased ingredient information" is data regarding the types and quantities of ingredients actually purchased by the user.

[0118] "Means for providing cooking instructions" refers to methods or services that allow users to refer to specific cooking procedures.

[0119] "Leftover ingredient information" is data regarding the type and amount of ingredients that were not used after cooking.

[0120] As an embodiment of the present invention, we will specifically explain a system that generates meal menus based on a user's health information and home food inventory information, procures the necessary ingredients through an online sales service, and provides cooking instructions. This system is composed of a server, terminals, and users.

[0121] First, when a user uses the system for the first time, they use their device to enter their own health information (e.g., age, gender, allergy information, calorie intake goal). The device sends this information to the server using an HTTP request, and the server stores the received information in a database. Next, the user enters information about food ingredients in their home (e.g., rice, chicken, vegetables, etc.), and the device again sends this to the server for storage. The database used can be, for example, MySQL.

[0122] Next, the server uses a generative AI model (e.g., OpenAI's GPT-4) to generate a meal menu for several days based on the stored health information and food inventory information. For example, if the calorie intake goal is 1500 kcal, a menu such as "chicken and tomato soup, salad, and brown rice" can be generated. The generated menu is sent from the server to the user's device.

[0123] The device generates a list of required ingredients (e.g., "300g chicken, 2 tomatoes, salad vegetables, 2 cups brown rice") based on the received menu and displays it to the user. When the user checks and edits the list of ingredients and taps the "Order" button, the device sends the order data using the API of the online sales service. For example, a general REST API can be used here. The online sales service receives the order and sets the delivery date and time and collection method.

[0124] Once the order details are confirmed, the server generates a QR code based on that information and sends it to the user's device. The user then uses this QR code to confirm receipt of the item through the online sales service. QR codes can be generated using the zxing library, for example.

[0125] The user enters the received ingredient information into the device, which then sends the information back to the server. The server stores the received data and retrieves specific cooking instructions and video links via the cooking assistant app's API (e.g., a cooking recipe service). The retrieved information is then sent from the server to the user's device and displayed.

[0126] Furthermore, the server analyzes and records leftover ingredient information based on the user's purchase and cooking history. By incorporating this information into the next menu generation, it is possible to create recipes that minimize food waste. For example, it can suggest new salad recipes using unused tomatoes.

[0127] Examples and prompts

[0128] As a concrete example, let's consider the case of a female user in her 30s using this system for the first time. First, the user enters her profile (e.g., female in her 30s, no allergies, calorie intake goal 1500 kcal) and household ingredient inventory (e.g., rice, chicken, tomatoes) into her device. This information is saved on the server, and the generative AI model uses it to generate a menu of "chicken and tomato soup, salad, and brown rice" and sends it to the user's device. The user checks and edits the ingredient list and submits the order to the online sales service, where the order details are sent as a QR code. The user then re-enters the received ingredient information, and a link to a cooking video providing detailed cooking instructions is displayed on the user's device.

[0129] An example prompt is:

[0130] "Generate a healthy meal plan for a woman in her 30s with no allergies and a calorie intake goal of 1500 kcal. The ingredients in the household are rice, chicken, and tomatoes."

[0131] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0132] Step 1: User registration and initial setup

[0133] The user launches the application and enters health information such as age, gender, allergy information, and calorie intake goal into the device. The entered information is sent by the device to the server via an HTTP request. The server stores the received health information in a database (e.g., MySQL). Next, the user enters the food inventory in the home (e.g., rice, chicken, vegetables, etc.), and the device similarly sends this to the server and stores it in the database.

[0134] Input: age, gender, allergy information, calorie intake goal, food inventory information.

[0135] Output: Health information and food inventory information stored in a database.

[0136] Specific operation: The user enters information into the input field of the smartphone app and taps the "Save" button. This operation causes the device to send the information to the server. The server receives the data and stores it in a database.

[0137] Step 2: Meal menu generation

[0138] The server retrieves the stored health information and food inventory information from the database and sends a prompt to a generative AI model (e.g., OpenAI's GPT-4). The generative AI model generates a meal menu (e.g., "chicken and tomato soup, salad, and brown rice") and sends it to the server. The server stores the received menu in a database and sends it to the user's device.

[0139] Input: Health information, food inventory information, prompt text.

[0140] Output: The generated meal menu.

[0141] Specific operation: The server creates a prompt and sends it to the AI ​​model. The AI ​​model generates a menu and returns it to the server. The server saves the menu and sends it to the device. The user's device displays the menu via a push notification.

[0142] Step 3: Generate materials list and order

[0143] Based on the menu received, the terminal generates a list of required ingredients and displays it to the user. The user checks the list of ingredients, makes any necessary edits (e.g., changes to quantities, adding or deleting items), and confirms the "order." The terminal then sends the order data via the API of the online sales service. The online sales service receives the order data and sets the delivery date and time, etc.

[0144] Input: The generated meal menu.

[0145] Output: Generated material list, order details.

[0146] Specific operation: The terminal analyzes the menu and generates an ingredient list. The user checks and edits the list. After confirming the order, the terminal sends the order to the online sales service's API. The online sales service receives and processes the order.

[0147] Step 4: Generate and distribute QR codes

[0148] Once the order details are confirmed, the server generates a QR code based on the order information and sends it to the user's device. The user then uses this QR code when picking up the item from the online shopping service.

[0149] Input: Order details.

[0150] Output: QR code.

[0151] Specific operation: The server analyzes the order details and generates a QR code (e.g., using the zxing library). The generated QR code URL is sent to the user's device. The user displays the QR code on their device and uses it to receive the item.

[0152] Step 5: Save purchased ingredients and provide cooking instructions

[0153] The user enters the received ingredient information into the device and sends it to the server. The server stores the received information in a database and obtains specific cooking instructions and video links via the cooking assistant app's API (e.g., a cooking recipe service). The server sends this information to the user's device, which then displays it to the user.

[0154] Input: Received ingredient information.

[0155] Output: Specific cooking instructions and video links.

[0156] Specific operation: The user enters the ingredients they have purchased into their device and taps the "Save" button. The device sends the information to the server. The server saves it in a database, then calls the cooking app's API to retrieve the cooking information. The retrieved cooking instructions are sent to the user's device, which displays them to the user.

[0157] Step 6: Manage leftover ingredients and create your next recipe

[0158] The server analyzes and records leftover ingredient information based on the purchase and cooking history sent by the user. This information is reflected in the next menu generation and suggests recipes that minimize waste.

[0159] Input: purchase history, cooking history.

[0160] Output: Information on leftover ingredients, next recipe.

[0161] Specific operation: The server retrieves historical information from the database and uses the AI ​​model to calculate the remaining amount of unused ingredients. The calculation result is included in the next prompt and sent to the AI ​​model. The model generates a new menu, which the server saves in the database. The generated menu is then notified to the user's device.

[0162] (Application example 1)

[0163] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0164] In modern society, personal health management and efficient dietary habits are important issues. In particular, the lack of a system that centrally manages multiple elements, such as generating efficient meal menus based on health information and food inventory information, online ordering, and providing cooking instructions, is a problem. There is also a need for a method to provide menus optimized for each individual's health status and use ingredients without waste based on such menus.

[0165] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0166] In this invention, the server includes means for inputting and saving personal information including health information, means for inputting and saving information on household food inventory, means for generating a meal menu using the saved personal information and food ingredient information, means for displaying a list of necessary ingredients based on the generated meal menu, means for ordering ingredients via an online supermarket API, means for converting the order content into a QR code and distributing it, means for saving information on purchased ingredients and providing cooking instructions, means for recording information on leftover ingredients and reflecting it in the next menu generation, means for providing instructions to a smartphone app based on the generated meal menu and food ingredient information, means for generating prompts, and means for generating an optimal meal menu using a generative AI model. This enables the provision of individually optimized healthy menus, the use of ingredients without waste, and centralized management of online orders.

[0167] "Health information" refers to information related to an individual's health, such as an individual's age, gender, allergy information, and calorie intake goal.

[0168] "Personal information" is information necessary to identify a specific individual, including health information.

[0169] "Food inventory information" refers to information about the types and amounts of food ingredients currently stored in the home.

[0170] "Meal Menu" refers to a combination of dishes or a menu suggested based on personal health information and food inventory information.

[0171] "Ingredients list" refers to a list of the types and quantities of ingredients needed to prepare a particular meal.

[0172] An "online supermarket" is an online retail store where you can order and purchase food and daily necessities via the Internet.

[0173] "API" is an abbreviation for Application Program Interface, and refers to an interface for linking functions between software.

[0174] "QR code" refers to a two-dimensional barcode that allows for easy electronic acquisition of information.

[0175] "Cooking instructions" refers to information such as steps or tips for making a particular dish, or a video link showing how to do so.

[0176] A "prompt" is a message or instruction that requests some input or response from the user.

[0177] A "generative AI model" refers to an artificial intelligence model that generates optimal output based on input data.

[0178] "Smartphone app" refers to a software application that runs on a smartphone.

[0179] As a form for implementing this invention, we will specifically explain a series of systems that generate meal menus based on health information and ingredient inventory information, procure ingredients through collaboration with an online supermarket, and provide cooking methods.

[0180] User registration and initial settings

[0181] When a user uses the system for the first time, they first enter their profile information (age, gender, allergy information, calorie intake goal, etc.) into the device. The device sends the entered information to the server, which stores it. Next, the user enters information about food ingredients in their home (e.g., rice, chicken, vegetables, etc.) into the device, which also sends this information to the server and stores it.

[0182] Meal Menu Generation

[0183] The server retrieves the stored personal information and food inventory information and uses the generative AI model to generate several days' worth of meal menus. For example, it generates a menu of "chicken and tomato soup, salad, and brown rice" based on a calorie intake target of 1500 kcal. The generated menu is sent from the server to the user's device.

[0184] Generate and order materials lists

[0185] Based on the received menu, the terminal displays a list of required ingredients (e.g., "300g chicken, 2 tomatoes, salad vegetables, 2 cups brown rice") to the user. The user can check this list and edit it as necessary (for example, change the amount, add, or delete). The confirmed ingredient list is sent as an order from the terminal via the online supermarket's API. The online supermarket receives the order and sets the delivery date, time, and collection method.

[0186] QR Code Generation and Distribution

[0187] Once the order details are confirmed, the server converts the information into a QR code and sends it to the user's device. The user can then use the QR code received on their device to confirm receipt at the online supermarket.

[0188] Save purchased ingredients information and provide cooking instructions

[0189] The user inputs the delivered ingredients into the device and sends that information to the server. The server saves the purchased ingredient information and, based on the saved data, retrieves specific cooking instructions and video links via the cooking app's API. This information is sent from the server to the user's device, and the user can cook while referring to the cooking instructions displayed on the device.

[0190] Managing leftover ingredients and creating your next recipe

[0191] The server analyzes and records leftover ingredients based on the user's purchase and cooking history. This information is reflected in the next menu generation and creates recipes to minimize waste. For example, it might suggest a new salad recipe using unused tomatoes.

[0192] Specific examples

[0193] When a female user in her 30s uses the system for the first time, she first enters her profile (e.g., female in her 30s, no allergies, calorie intake goal 1500 kcal) and her household food inventory (e.g., rice, chicken, tomatoes) into the device. Once this information is saved on the server, the generative AI model uses it to create a menu of "chicken and tomato soup, salad, brown rice" and sends it to the user.

[0194] Once the user checks and edits the ingredient list and submits the order to the online supermarket, the order details are sent as a QR code. After that, the user re-enters the received ingredient information, and a link to a cooking video providing detailed cooking instructions is displayed on the device.

[0195] Prompt Sentence Examples

[0196] User Information:

[0197] Age: 30

[0198] Gender: Female

[0199] Calorie intake goal: 1500 kcal

[0200] Allergy Information: None

[0201] Ingredients availability information:

[0202] Rice: 2 cups

[0203] Chicken: 1 piece

[0204] Tomatoes: 3

[0205] Please suggest a healthy meal plan based on the above information.

[0206] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0207] Step 1:

[0208] The user enters their profile information into the device, which then sends the information (such as age, gender, allergy information, and calorie intake goal) to the server, where it is stored in a database.

[0209] Input: Age, gender, allergy information, calorie intake goal

[0210] Output: Personal information stored in the database

[0211] Specific operation: The user enters information using a smartphone app and presses the "Save" button. The device sends the entered information as a request to the server's API, and the server saves the information in a database.

[0212] Step 2:

[0213] The user inputs information about the food inventory in their home into the terminal. The input data (e.g., rice, chicken, tomatoes, etc.) is sent from the terminal to the server, which then stores it in a database.

[0214] Input: Ingredient information such as rice, chicken, and tomatoes

[0215] Output: Ingredient inventory information stored in the database

[0216] Specific operation: The user enters the inventory and presses the "Save" button. The terminal sends the information as a request to the server's API, and the server saves the information in the database.

[0217] Step 3:

[0218] The server retrieves the stored personal information and food inventory information and generates several days' worth of meal menus using a generative AI model.

[0219] Input: Personal information, food inventory information

[0220] Output: Generated meal menus for several days

[0221] Specific operation: The server inputs personal information and food inventory information into the AI ​​model and generates the following prompt: "User information: Age: 30, Gender: Female, Calorie intake goal: 1500 kcal, Allergy information: None, Food inventory information: Rice: 2 cups, Chicken: 1, Tomatoes: 3. Based on the above information, please suggest a healthy meal menu." This is input into the generative AI model to obtain a meal menu.

[0222] Step 4:

[0223] Based on the received menu, the device displays a list of ingredients (e.g., "300g chicken, 2 tomatoes, salad greens, 2 cups brown rice") to the user, who can then review and edit the list if necessary.

[0224] Input: Generated meal menu

[0225] Output: Materials list

[0226] Specific operation: The terminal retrieves and displays the menu, and the user confirms and edits it. The terminal visualizes and displays the ingredient list.

[0227] Step 5:

[0228] The confirmed ingredients list is then sent from the device to the online supermarket via its API, which then receives the order and sets the delivery date and time and pick-up method.

[0229] Input: Edited materials list

[0230] Output: Order information and delivery date / time / pickup method settings

[0231] Specific operation: The terminal sends an order request to the online supermarket's API, the server processes the order, and the terminal displays an order confirmation.

[0232] Step 6:

[0233] Once the order details are confirmed, the server converts the information into a QR code and sends it to the user's device. The user then uses the QR code received on their device to confirm receipt at the online supermarket.

[0234] Input: Order Information

[0235] Output: QR code

[0236] Specific operation: The order information is converted using a QR code generation library, and the generated QR code is sent to the device. The user scans the QR code when receiving the order.

[0237] Step 7:

[0238] The user inputs the delivered ingredients into the device and sends the information to the server. The server saves the purchased ingredients and retrieves specific cooking instructions and video links based on the saved data via the cooking app's API.

[0239] Input: Delivered food information

[0240] Output: Cooking instructions and video links

[0241] Specific operation: The user inputs the delivered ingredients, and the device sends the information to the server. The server accesses the cooking app's API to retrieve the information and displays it on the user's device.

[0242] Step 8:

[0243] The server analyzes and records leftover ingredients based on the purchase and cooking history sent by the user. This information is reflected in the next menu generation, and recipes are created to minimize waste.

[0244] Input: purchase history, cooking history

[0245] Output: Recipe reflected in next menu generation

[0246] Specific operation: The server analyzes purchase and cooking history data, records leftover ingredients, and reflects them in the next menu generation. If necessary, it generates new prompts and inputs them into the generative AI model to obtain recipes.

[0247] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0248] As a form for implementing the present invention, we will specifically explain a system that generates meal menus based on health information and ingredient inventory information, procures ingredients through collaboration with an online supermarket, and provides cooking methods, all of which are combined with an emotion engine that recognizes the user's emotions.

[0249] User registration and initial settings

[0250] When a user uses the system for the first time, they first enter their profile information (age, gender, allergy information, calorie intake goal, etc.) into the device. The device then sends the entered information to the server, which stores it. The user then enters information about the food ingredients in their home (e.g., rice, chicken, vegetables, etc.) into the device, which then sends this information to the server and stores it. In addition, an interface that allows the emotion engine to collect the user's emotion information is also included as part of the initial setup.

[0251] Manipulating the Emotion Engine

[0252] The emotion engine continuously collects data such as the user's facial expressions, voice, and text messages to recognize their emotions, and then analyzes and transmits emotional information such as their current mood and stress level to the server.

[0253] Meal Menu Generation

[0254] The server uses an AI model to generate several days' worth of meal menus based on stored personal information, ingredient inventory information, and emotional information provided by the emotion engine. For example, if the server detects that the user is feeling stressed, it generates a menu using ingredients that have a relaxing effect. The generated menu is then sent from the server to the user's device.

[0255] Generate and order materials lists

[0256] Based on the received menu, the terminal displays a list of required ingredients (e.g., "300g chicken, 2 tomatoes, salad vegetables, 2 cups brown rice") to the user. The user can check this list and edit it as necessary (e.g., change the amount, add, or delete). The confirmed ingredient list is sent as an order from the terminal via the online supermarket's API. The online supermarket receives the order and sets the delivery date and time and collection method.

[0257] QR Code Generation and Distribution

[0258] Once the order details are confirmed, the server converts the information into a QR code and sends it to the user's device. The user can then use the QR code received on their device to confirm receipt at the online supermarket.

[0259] Save purchased ingredients information and provide cooking instructions

[0260] The user inputs the delivered ingredients into the device and sends that information to the server. The server saves the purchased ingredient information and, based on the saved data, retrieves specific cooking instructions and video links via the cooking app's API. This information is sent from the server to the user's device, and the user can cook while referring to the cooking instructions displayed on the device.

[0261] Managing leftover ingredients and creating your next recipe

[0262] The server analyzes and records leftover ingredient information based on the user's purchase and cooking history. Furthermore, this information, along with data from the emotion engine, is taken into account when generating the next menu, creating recipes to minimize waste. For example, it can suggest new salad recipes using unused tomatoes.

[0263] Specific examples

[0264] A female user in her 30s uses the system for the first time and enters her profile information (e.g., female in her 30s, no allergies, calorie intake goal 1500 kcal) and her household food inventory (e.g., rice, chicken, tomatoes) into the device. The emotion engine detects that the user is feeling stressed. This information is also sent to the server, and the AI ​​uses it to generate a menu of "relaxing chicken and tomato soup, salad, and brown rice" and sends it to the user.

[0265] Once the user confirms and edits the ingredient list and submits the order to the online supermarket, the order details are sent as a QR code. When the user re-enters the received ingredient information, a link to a cooking video with specific cooking instructions is displayed on the device. The system aims to simplify daily meal preparation and promote health maintenance and efficient ingredient management.

[0266] The present invention reduces the burden of meal planning and food ingredient management on the user, and further makes it possible to propose menus that match the user's mood based on emotional information, thereby enabling the user to maintain a healthy diet.

[0267] The processing flow will be explained below.

[0268] Step 1:

[0269] The user enters profile information (age, gender, allergy information, calorie intake goal, etc.) into the terminal.

[0270] Step 2:

[0271] The terminal transmits the entered profile information to the server.

[0272] Step 3:

[0273] The server stores the received profile information in a database.

[0274] Step 4:

[0275] The user inputs information about food ingredients in the home into the terminal.

[0276] Step 5:

[0277] The terminal transmits the input food stock information to the server.

[0278] Step 6:

[0279] The server stores the received food stock information in a database.

[0280] Step 7:

[0281] The emotion engine collects the user's emotional information (facial expressions, voice, text messages, etc.) and recognizes emotions.

[0282] Step 8:

[0283] The emotion engine sends the recognized emotion information to the server.

[0284] Step 9:

[0285] The server acquires the stored personal information, food stock information, and emotion information.

[0286] Step 10:

[0287] The server uses an AI model to generate a meal menu for several days that takes into account emotional information. For example, if the user is feeling stressed, it will generate a menu with a relaxing effect.

[0288] Step 11:

[0289] The server transmits the generated meal menu to the user terminal.

[0290] Step 12:

[0291] The terminal displays the meal menu received from the server to the user.

[0292] Step 13:

[0293] The user reviews the meal menu and edits the ingredient list as needed.

[0294] Step 14:

[0295] The terminal sends the confirmed list of ingredients to the online supermarket's API.

[0296] Step 15:

[0297] The online supermarket processes the order based on the received ingredient list and sets the delivery date and time and collection method.

[0298] Step 16:

[0299] The server receives the order details from the online supermarket and converts them into a QR code.

[0300] Step 17:

[0301] The server sends the generated QR code to the user's terminal.

[0302] Step 18:

[0303] The user receives the QR code on their device and uses it to receive the ingredients.

[0304] Step 19:

[0305] The user inputs the delivered food information into the terminal.

[0306] Step 20:

[0307] The terminal transmits information about the delivered ingredients to the server.

[0308] Step 21:

[0309] The server stores the received purchased ingredient information in a database.

[0310] Step 22:

[0311] Based on the saved data, the server obtains specific cooking instructions and video links via the cooking app's API.

[0312] Step 23:

[0313] The server sends the acquired cooking instructions and video links to the user's terminal.

[0314] Step 24:

[0315] The user cooks the food while referring to the cooking instructions displayed on the terminal.

[0316] Step 25:

[0317] The server analyzes the purchase history and cooking history sent by the user and records information about leftover ingredients.

[0318] Step 26:

[0319] The server takes into account the remaining ingredients and emotion information and reflects them in the next menu generation. For example, it generates a menu using unused tomatoes.

[0320] This series of processing steps streamlines the user's meal preparation and health management, while providing a balanced meal plan based on emotional information.

[0321] Example 2

[0322] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0323] Conventional meal planning systems only considered personal health information and household food inventory information, and were unable to suggest menus that took the user's emotional state into account. This made it difficult to suggest appropriate menus when the user was feeling stressed or in a particular mood, limiting their ability to provide comprehensive health management for the user. Furthermore, their management of food waste to a minimum was insufficient, leaving a need for more efficient food use.

[0324] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0325] In this invention, the server includes means for inputting and saving personal information including health information, means for inputting and saving household ingredient inventory information, means for generating a meal menu using the saved personal information and ingredient inventory information, means for displaying a list of ingredients required based on the generated meal menu, means for ordering ingredients via an API of an online sales service, means for converting the order details into a QR code and distributing it, means for saving information on purchased ingredients and providing cooking instructions, means for recording information on leftover ingredients and reflecting it in generating the next menu, and means for collecting, analyzing information on the user's emotions and reflecting it in generating the meal menu. This makes it possible to propose an appropriate meal menu based on the user's health condition and emotional state, and to reduce waste by using ingredients efficiently.

[0326] "Health information" is data about the user's physical and health status, such as age, gender, allergy information, and calorie intake goals.

[0327] "Personal Information" is any data about a user, including profile information and health information.

[0328] "Home food ingredient inventory information" is information about the types and quantities of ingredients currently stored in the user's home.

[0329] A "meal menu" is a list of meals that the user plans to consume, generated based on the user's personal information and home food inventory information.

[0330] The "ingredient list" is a list of ingredients required based on the generated meal menu.

[0331] An "API for online sales services" is a program interface for ordering products online.

[0332] A "QR code" is a two-dimensional code that visually conveys information about an order.

[0333] "Purchased ingredient information" is information about ingredients purchased by a user via an online sales service.

[0334] "Cooking instructions" is information about the steps and techniques for cooking using purchased ingredients.

[0335] "Leftover ingredient information" is information about ingredients that were left over from the previous cooking session and were not used.

[0336] "Emotion information" is data relating to the user's emotional state obtained from facial expressions, voice, text, etc.

[0337] The present invention combines a series of systems that generate meal menus based on a user's health information and household food inventory information, procure ingredients in cooperation with online sales services, and provide cooking methods with an emotion engine that recognizes the user's emotion information. Specific embodiments of this system are described below.

[0338] First, when using the device for the first time, the user enters their profile information (age, gender, allergy information, calorie intake goal, etc.) into the device, which then sends and stores this information on the server. Next, the user enters information about the food ingredients in their home (rice, chicken, tomatoes, etc.) into the device, which then sends this information to the server and stores it. The initial settings for the emotion engine are also performed on the device, and an interface is built to collect the user's emotion information.

[0339] The emotion engine collects emotional data from users through facial expression analysis, voice recognition, and text analysis, and analyzes it in real time. The analysis results are sent to a server to identify their state of stress and other factors.

[0340] The server combines the stored personal information, food inventory information, and emotional information provided by the emotion engine to generate a meal menu for several days using an AI model. For example, if the server determines that the user is feeling stressed, it will suggest a menu using ingredients that have a relaxing effect. This generated menu is then sent from the server to the user's device.

[0341] The user's device creates a list of ingredients based on the received menu and displays it to the user. Once the user confirms and edits the ingredients list, the confirmed list is sent to the online supermarket via its API. The online supermarket receives the order and sets the delivery date and time and collection method.

[0342] Once the order details are confirmed, the server converts the information into a QR code and sends it to the user's device. The user uses the QR code on their device to confirm receipt at the online supermarket. Information about the delivered ingredients is again entered into the device and sent to the server. Based on this information, the server obtains specific cooking instructions and a link to a cooking video via the cooking assistance application's API and sends it to the user's device. The user cooks while referring to this information.

[0343] The server also analyzes the user's purchase and cooking history and records any leftover ingredients. The next time the server generates a menu, it takes this information and emotional information into account to create recipes that minimize waste. For example, it might suggest a new salad recipe using unused tomatoes.

[0344] The present invention makes it possible to propose an appropriate meal menu according to the user's health condition and emotional state, and also reduces waste by using ingredients more efficiently.

[0345] Specific examples

[0346] A female user in her 30s uses the system for the first time and enters her profile information (e.g., female in her 30s, no allergies, calorie intake goal 1500 kcal) and her household food inventory (e.g., rice, chicken, tomatoes) into the device. The emotion engine detects that the user is feeling stressed. This information is also sent to the server, and the AI ​​uses it to generate a menu of "relaxing chicken and tomato soup, salad, and brown rice" and sends it to the user.

[0347] Prompt Sentence Examples

[0348] "A woman in her 30s with no allergies and a calorie intake goal of 1500 kcal. She has rice, chicken, and tomatoes at home. Please suggest a meal menu that will help the user relax when they are feeling stressed."

[0349] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0350] Step 1: User registration and initial setup

[0351] When a user uses the device for the first time, they enter their profile information (e.g., age, gender, allergy information, calorie intake goal, etc.). The entered information is sent from the device to the server, which stores it in a database. Next, the user enters information about the food inventory in their home (e.g., 1 kg of rice, 500 g of chicken, 3 tomatoes) into the device, which also sends this information to the server and stores it. The device then initializes the emotion engine and builds an interface to collect the user's emotion information.

[0352] input:

[0353] Profile information (age, gender, allergy information, calorie intake goal, etc.)

[0354] Home food inventory information (rice, chicken, tomatoes, etc.)

[0355] output:

[0356] Personal information and food inventory information stored on the server

[0357] The initial settings of the emotion engine are complete

[0358] Specific behavior:

[0359] The terminal receives input information from the user and sends it to the server via the API.

[0360] The server stores the received data in a database.

[0361] The terminal displays an emotion engine setting wizard and asks the user for permission to collect emotion data.

[0362] Step 2: Manipulating the Emotion Engine

[0363] The emotion engine collects emotion data through facial expression analysis, voice recognition, and text analysis. This data is analyzed in real time to extract the user's current emotion information (e.g., stress level) and send it to the server.

[0364] input:

[0365] User facial expressions, voice, and text data

[0366] output:

[0367] Analyzed emotional information (e.g., whether the user is feeling stressed)

[0368] Specific behavior:

[0369] The emotion engine captures the user's facial expressions and voice via the camera and microphone and analyzes text messages.

[0370] The emotion engine uses machine learning algorithms to analyze the emotion data and send the results to a server.

[0371] The server integrates the received emotion information into a user profile and stores it.

[0372] Step 3: Meal menu generation

[0373] The server uses an AI model to generate a meal menu based on stored personal information, food inventory information, and emotional information. For example, if the server detects that the user is feeling stressed, it will generate a menu using ingredients that have a relaxing effect. The generated menu is then sent from the server to the user's device.

[0374] input:

[0375] personal information

[0376] Food inventory information

[0377] emotional information

[0378] output:

[0379] Generated meal menu (e.g., relaxing chicken and tomato soup, salad, and brown rice)

[0380] Specific behavior:

[0381] The server inputs the stored data into an AI model, which then generates a meal menu.

[0382] The server sends the generated menu to the terminal.

[0383] Step 4: Generate materials list and order

[0384] The device creates a list of required ingredients based on the received menu and displays it to the user. The user can check this list and change the quantities or add or delete items. The confirmed list is then sent from the device via the API of the online sales service. The online sales service receives the order details and sets the delivery date and time and collection method.

[0385] input:

[0386] Generated meal menu

[0387] output:

[0388] Required Materials List

[0389] Submitted order information

[0390] Specific behavior:

[0391] The device analyzes the meal menu and automatically generates a list of necessary ingredients.

[0392] The user edits this list, and after reviewing it, sends it to the online sales service via API.

[0393] The online sales service receives the order information and begins processing it.

[0394] Step 5: Generate and distribute QR codes

[0395] The server converts the order details into a QR code and sends it to the user's device, which the user can use to confirm receipt of the order through the online sales service.

[0396] input:

[0397] Order details

[0398] output:

[0399] QR code

[0400] Specific behavior:

[0401] The server encodes the order details into a QR code and sends the generated QR code to the terminal.

[0402] The user displays the QR code on their device and scans it upon receipt.

[0403] Step 6: Save purchased ingredients and provide cooking instructions

[0404] The user inputs the delivered ingredients into the device and sends that information to the server. The server saves the purchased ingredient information and, based on the saved data, retrieves specific cooking instructions and video links via the cooking assistance application's API. This information is sent from the server to the user's device, and the user cooks while referring to the cooking instructions displayed on the device.

[0405] input:

[0406] Purchased food information

[0407] output:

[0408] Saved ingredient information

[0409] Cooking methods and cooking video links

[0410] Specific behavior:

[0411] The user enters the received ingredient information into the terminal and sends it to the server via the API.

[0412] The server uses the stored ingredient information to connect to the API of the cooking assistance application and obtain cooking instructions and video links.

[0413] The acquired information is sent to the terminal so that the user can view and implement it.

[0414] Step 7: Manage leftover ingredients and create your next recipe

[0415] The server analyzes and records leftover ingredient information based on the purchase and cooking history sent by the user. Taking this information and data from the emotion engine into consideration, the server generates recipes that minimize waste and are reflected in the next menu creation.

[0416] input:

[0417] Purchase history

[0418] Cooking history

[0419] emotional information

[0420] output:

[0421] Leftover food information

[0422] Menu for next time

[0423] Specific behavior:

[0424] The server analyzes the data sent by the user and identifies any leftover ingredients.

[0425] The remaining ingredients will be taken into consideration when creating the next recipe, and a menu will be created to minimize waste.

[0426] The new menu is sent to the user terminal.

[0427] (Application example 2)

[0428] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0429] In modern life, it is extremely difficult to efficiently manage daily meal preparation while maintaining a healthy diet. In particular, there is a demand for home food ingredient management and flexible menu suggestions that respond to the user's mood. Furthermore, while meeting these demands, it is also necessary to achieve smooth food procurement through integration with online supermarkets. To solve these issues, it is essential to develop a system that can centrally manage the entire process and is easy for users to use.

[0430] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0431] In this invention, the server includes means for inputting and saving personal information including health information, means for inputting and saving information about home food ingredient inventory, means for generating a meal menu using the saved personal information and ingredient information, means for displaying a list of ingredients needed based on the generated meal menu, means for ordering ingredients via an online supermarket API, means for converting the order details into a QR code and distributing it, means for saving information about purchased ingredients and providing cooking instructions, means for recording information about remaining ingredients and reflecting this in generating the next menu, means for recognizing and collecting user emotional information, means for analyzing the emotional information and optimizing the meal menu based on the analysis results, means for acquiring the emotional information on a smartphone, and means for displaying the suggested meal menu via an application used in the virtual store. This allows for the suggestion of an optimal meal menu based on the user's health condition and emotions, allowing for smooth procurement of ingredients and efficient cooking based on the menu.

[0432] "Health information" refers to information related to the user's health condition and nutritional balance, such as age, gender, allergy information, and calorie intake goal.

[0433] "Personal Information" refers to information that is individually associated with a user, specifically the user's profile information, health information, and emotional information.

[0434] "Inventory information for ingredients" refers to information indicating the types and quantities of ingredients currently available in the home, such as rice, chicken, and vegetables.

[0435] "Emotional information" refers to information that indicates the user's current emotional state, such as stress level and mood.

[0436] "Virtual store" refers to a platform that provides a virtual store experience via the Internet, where users can purchase products.

[0437] "Emotion engine" refers to the algorithms and mechanisms for recognizing and collecting a user's emotional state by analyzing data such as facial expressions, voice, and text messages.

[0438] A "generative AI model" refers to an artificial intelligence algorithm that provides a rational output based on input data. Examples include GPT-4 and BERT.

[0439] A "prompt sentence" refers to the input sentence that is fed into a generative AI model, which then generates an output based on that sentence.

[0440] "Diet Menu" refers to a list of meals that are recommended for the user to consume, generated based on health and emotional information.

[0441] "Ingredient list" refers to a list of ingredients needed to cook a meal based on the menu. For example, "300g chicken, 2 tomatoes."

[0442] "Net supermarket" refers to a retail service that allows users to order food and daily necessities online. Users can purchase the products they need via the Internet.

[0443] An "API" is an interface that allows different software systems to share functionality. Specifically, you submit an order through the online supermarket's API.

[0444] A "QR code" is a two-dimensional barcode that visually encodes digital information and can be scanned with a smartphone or other device.

[0445] "Cooking method" refers to the steps for making a dish using specified ingredients. Specifically, it can be a recipe or a link to a cooking video.

[0446] A "cooking app" refers to an application that provides cooking methods and recipes that users can refer to to prepare a dish.

[0447] User registration and initial settings

[0448] When a user uses the system for the first time, they enter their profile information (age, gender, allergy information, calorie intake goal, etc.) into the device using their smartphone, and this information is sent to the server. Next, they enter information about their household food inventory (e.g., rice, chicken, vegetables, etc.) into the device, and this information is also sent and saved to the server. An interface that allows the emotion engine to collect the user's emotion information is also included as part of the initial setup.

[0449] Manipulating the Emotion Engine

[0450] The emotion engine uses the smartphone's camera and microphone to continuously collect the user's facial expressions and voice to recognize their emotions. This information is then analyzed to determine their current mood and stress level, and the results are sent to the server. The user's emotional data includes stress, relaxation, happiness, etc.

[0451] Generate a meal menu

[0452] The server generates several days' worth of meal menus using a generative AI model (e.g., GPT-4, BERT) based on stored personal information, ingredient inventory information, and emotional information provided by the emotion engine. For example, if the server recognizes that the user is feeling stressed, it generates a menu using ingredients that have a relaxing effect. The generated menu is sent from the server to the user's device.

[0453] Generate and order materials lists

[0454] The terminal displays a list of ingredients required based on the received menu (e.g., "300g chicken, 2 tomatoes, salad vegetables, 2 cups brown rice") to the user. The user can check and edit this list, and an order is placed using the online supermarket's API. The online supermarket receives the order and delivers it at the specified date and time.

[0455] QR Code Generation and Distribution

[0456] Once the order details are confirmed, the server converts the information into a QR code and sends it to the user's device. The user then uses the QR code received on their device to confirm receipt at the online supermarket.

[0457] Save purchased ingredients information and provide cooking instructions

[0458] The user inputs the delivered ingredients into the device and sends that information to the server. The server saves the purchased ingredient information and obtains specific cooking instructions and cooking video links via the cooking app's API. This information is sent from the server to the user's device, and the user can cook while referring to the cooking instructions displayed on the device.

[0459] Managing leftover ingredients and creating your next recipe

[0460] The server analyzes and records leftover ingredient information based on the purchase and cooking history sent by the user. Taking this information and data from the emotion engine into consideration, the server generates recipes to minimize waste and incorporates it into the next menu creation. For example, it can suggest new salad recipes using unused tomatoes.

[0461] Specific examples

[0462] A female user in her 30s uses the system for the first time and enters her profile information (e.g., female in her 30s, no allergies, calorie intake goal 1500 kcal) and her household food inventory (e.g., rice, chicken, tomatoes) into her device. The emotion engine detects that the user is feeling stressed, and this information is also sent to the server. Based on this, the generative AI model generates a menu of "relaxing chicken and tomato soup, salad, and brown rice" and sends it to the user. When the user checks and edits the ingredient list and submits the order to the online supermarket, the order details are sent as a QR code. When the received ingredient information is entered again, a link to a cooking video providing specific cooking instructions is displayed on the device.

[0463] Prompt Sentence Examples

[0464] For example, if the emotion engine detects that the user is feeling "stressed," the prompt for the AI ​​might look like this:

[0465] Prompt text:"

[0466] Generate a relaxing meal plan for a 30-year-old female with the following food inventory: rice, chicken, and tomatoes. The user is currently feeling stressed."

[0467] In this way, the application supports a healthy lifestyle by providing optimal meal menus tailored to the user's health condition and mood. This system simplifies daily meal preparation, enabling users to lead an efficient and healthy diet.

[0468] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0469] Step 1:

[0470] User registration and initial setup:

[0471] A user uses a smartphone to input profile information (age, gender, allergy information, calorie intake goal, etc.) into the device. The device sends this information to the server, which stores the personal information in a database. The user also enters information about the household's food inventory (e.g., rice, chicken, vegetables, etc.) into the device, which also sends and stores this information in the server. The device then sets up an interface for the emotion engine, enabling it to collect the user's emotional information. At this point, the input is the user's profile information and food inventory information, and the output is the information stored in the server's database.

[0472] Step 2:

[0473] Emotion Recognition with Emotion Engine:

[0474] The emotion engine collects the user's facial expressions and voice in real time from the smartphone's camera and microphone and analyzes the emotional data. This allows the emotion engine to recognize emotional information such as the user's stress level and mood, and sends the analysis results to the server. The input to this step is data from the camera and microphone, and the output is analyzed emotional information.

[0475] Step 3:

[0476] Generate a meal menu:

[0477] The server uses a generative AI model to generate several days' worth of meal menus based on the stored personal information, ingredient inventory information, and emotional information. Specifically, the server supplies "prompt sentences" to the generative AI model to generate menus tailored to the user's emotions. The inputs for this step are personal information, ingredient inventory information, and emotional information, and the output is the generated meal menu.

[0478] Step 4:

[0479] Generate and display a materials list:

[0480] The server generates a list of ingredients based on the generated meal menu and sends it to the terminal. The terminal displays the ingredient list to the user, who can review and edit it. The input of this step is the generated meal menu, and the output is the displayed ingredient list.

[0481] Step 5:

[0482] Collaboration with online supermarkets:

[0483] The terminal sends the ingredient list that the user has confirmed and edited to the server, and the server automatically places the order via the online supermarket's API. Once the order details are confirmed, the server converts the order details into a QR code and sends it to the terminal. The input for this step is the confirmed and edited ingredient list, and the output is the QR-coded order information.

[0484] Step 6:

[0485] Receiving and storing materials:

[0486] The user inputs the delivered ingredients into the terminal and sends it to the server. The server stores this purchased ingredient information in a database. The input of this step is the purchased ingredient information entered by the user, and the output is the information stored in the database.

[0487] Step 7:

[0488] Cooking instructions provided:

[0489] The server obtains specific cooking instructions and video links based on the purchased ingredient information via the cooking app's API and sends them to the user's device. The device then displays the cooking instructions to the user. The input for this step is the purchased ingredient information, and the output is the provided cooking instructions and video links.

[0490] Step 8:

[0491] Manage leftover ingredients and create your next recipe:

[0492] The server analyzes and records leftover ingredient information based on the purchase history and cooking history sent by the user. Based on this, it generates recipes to minimize waste when generating the next menu. The input for this step is the purchase history and cooking history, and the output is the recipe reflected in the next menu generation.

[0493] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0494] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0495] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0496] [Second embodiment]

[0497] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0498] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0499] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0500] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0501] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0502] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0503] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0504] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0505] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0506] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0507] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0508] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0509] As a form for implementing the present invention, we will specifically explain a series of systems that generate meal menus based on health information and ingredient inventory information, procure ingredients through collaboration with an online supermarket, and provide cooking methods.

[0510] User registration and initial settings

[0511] When a user uses the system for the first time, they first enter their profile information (age, gender, allergy information, calorie intake goal, etc.) into the device. The device sends the entered information to the server, which stores it. Next, the user enters information about food ingredients in their home (e.g., rice, chicken, vegetables, etc.) into the device, which also sends this information to the server and stores it.

[0512] Meal Menu Generation

[0513] The server retrieves the stored personal information and food inventory information and uses the AI ​​model to generate a meal menu for several days. For example, it generates a menu of "chicken and tomato soup, salad, and brown rice" based on a calorie intake target of 1500 kcal. The generated menu is sent from the server to the user's device.

[0514] Generate and order materials lists

[0515] Based on the received menu, the terminal displays a list of required ingredients (e.g., "300g chicken, 2 tomatoes, salad vegetables, 2 cups brown rice") to the user. The user can check this list and edit it as necessary (e.g., change the amount, add, or delete). The confirmed ingredient list is sent as an order from the terminal via the online supermarket's API. The online supermarket receives the order and sets the delivery date and time and collection method.

[0516] QR Code Generation and Distribution

[0517] Once the order details are confirmed, the server converts the information into a QR code and sends it to the user's device. The user can then use the QR code received on their device to confirm receipt at the online supermarket.

[0518] Save purchased ingredients information and provide cooking instructions

[0519] The user inputs the delivered ingredients into the device and sends that information to the server. The server saves the purchased ingredient information and, based on the saved data, retrieves specific cooking instructions and video links via the cooking app's API. This information is sent from the server to the user's device, and the user can cook while referring to the cooking instructions displayed on the device.

[0520] Managing leftover ingredients and creating your next recipe

[0521] The server analyzes and records leftover ingredient information based on the user's purchase and cooking history. This information is reflected in the next menu generation and creates recipes to minimize waste. For example, it might suggest a new salad recipe using unused tomatoes.

[0522] Specific examples

[0523] When a female user in her 30s uses the system for the first time, she first enters her profile (e.g., female in her 30s, no allergies, calorie intake goal 1500 kcal) and her household food inventory (e.g., rice, chicken, tomatoes) into the device. Once this information is saved on the server, the AI ​​generates a menu of "chicken and tomato soup, salad, brown rice" based on that information and sends it to the user.

[0524] Once the user confirms and edits the ingredient list and submits the order to the online supermarket, the order is converted into a QR code and sent to them. After that, the user re-enters the received ingredient information, and a link to a cooking video with detailed cooking instructions is displayed on the device. The system aims to simplify daily meal preparation and promote health maintenance and efficient ingredient management.

[0525] The present invention reduces the burden of meal planning and food ingredient management on the user, enabling them to maintain a healthy diet.

[0526] The processing flow will be explained below.

[0527] Step 1:

[0528] The user enters profile information (age, gender, allergy information, calorie intake goal, etc.) into the terminal.

[0529] Step 2:

[0530] The terminal transmits the entered profile information to the server.

[0531] Step 3:

[0532] The server stores the received profile information in a database.

[0533] Step 4:

[0534] The user inputs information about food ingredients in the home into the terminal.

[0535] Step 5:

[0536] The terminal transmits the input food stock information to the server.

[0537] Step 6:

[0538] The server stores the received food stock information in a database.

[0539] Step 7:

[0540] The server uses an AI model to generate a meal menu for several days based on the saved profile information and food inventory information.

[0541] Step 8:

[0542] The server transmits the generated meal menu to the user terminal.

[0543] Step 9:

[0544] The terminal displays the meal menu received from the server to the user.

[0545] Step 10:

[0546] The user reviews the meal menu and edits the ingredient list as needed.

[0547] Step 11:

[0548] The terminal sends the confirmed list of ingredients to the online supermarket's API.

[0549] Step 12:

[0550] The online supermarket processes the order based on the received ingredient list and sets the delivery date and time and collection method.

[0551] Step 13:

[0552] The server receives the order details from the online supermarket and converts them into a QR code.

[0553] Step 14:

[0554] The server sends the generated QR code to the user's terminal.

[0555] Step 15:

[0556] The user receives the QR code on their device and uses it to receive the ingredients.

[0557] Step 16:

[0558] The user inputs the delivered food information into the terminal.

[0559] Step 17:

[0560] The terminal transmits information about the delivered ingredients to the server.

[0561] Step 18:

[0562] The server stores the received purchased ingredient information in a database.

[0563] Step 19:

[0564] The server connects to the cooking app's API based on the stored ingredient information to obtain specific cooking instructions and video links.

[0565] Step 20:

[0566] The server sends the acquired cooking instructions and video links to the user's terminal.

[0567] Step 21:

[0568] The user cooks the food while referring to the cooking instructions displayed on the terminal.

[0569] Step 22:

[0570] The server analyzes and records information about leftover ingredients based on the purchase history and cooking history sent by the user.

[0571] Step 23:

[0572] The server will take the remaining ingredient information into consideration when generating a new menu next time.

[0573] This series of processing steps streamlines meal preparation and health management for users.

[0574] Example 1

[0575] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0576] Conventional meal planning systems require users to plan their own meal menus, source the ingredients, and search for cooking methods, which is time-consuming. They also tend to waste ingredients and make efficient food management difficult. There is a need for a system that can solve these problems and allow users to easily maintain healthy meals.

[0577] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0578] In this invention, the server includes means for inputting and saving user information including health information, means for inputting and saving household food inventory information, means for generating a meal menu using a generative AI model based on the saved user information and food inventory information, means for displaying a list of necessary ingredients based on the generated meal menu, means for ordering ingredients via an interface of an online sales service, means for coding and distributing the order details, means for saving information on purchased ingredients and providing cooking instructions, and means for recording information on leftover ingredients and reflecting it in generating the next menu. This allows users to easily plan healthy meals and efficiently procure and manage the necessary ingredients.

[0579] "Health information" refers to health-related data such as a user's age, gender, allergy information, and calorie intake goal.

[0580] "User Information" means general data about individuals using the system, including health information.

[0581] "Food inventory information" is data on the types and quantities of food currently held in the household.

[0582] "Storage means" refers to the methods and techniques used to record and retain input information in a database.

[0583] A "generative AI model" is an algorithm or system that uses artificial intelligence technology to generate specific information from input data.

[0584] A "meal menu" is a list of specific meals that a user plans to consume.

[0585] An "ingredient list" is a list of ingredients and quantities needed to prepare a particular meal.

[0586] An "interface for online retail services" is a means of connection to an online retailer's system for ordering products.

[0587] "Encoding" refers to converting data into a format such as a barcode or QR code.

[0588] "Purchased ingredient information" is data regarding the types and quantities of ingredients actually purchased by the user.

[0589] "Means for providing cooking instructions" refers to methods or services that allow users to refer to specific cooking procedures.

[0590] "Leftover ingredient information" is data regarding the type and amount of ingredients that were not used after cooking.

[0591] As an embodiment of the present invention, we will specifically explain a system that generates meal menus based on a user's health information and home food inventory information, procures the necessary ingredients through an online sales service, and provides cooking instructions. This system is composed of a server, terminals, and users.

[0592] First, when a user uses the system for the first time, they use their device to enter their own health information (e.g., age, gender, allergy information, calorie intake goal). The device sends this information to the server using an HTTP request, and the server stores the received information in a database. Next, the user enters information about food ingredients in their home (e.g., rice, chicken, vegetables, etc.), and the device again sends this to the server for storage. The database used can be, for example, MySQL.

[0593] Next, the server uses a generative AI model (e.g., OpenAI's GPT-4) to generate a meal menu for several days based on the stored health information and food inventory information. For example, if the calorie intake goal is 1500 kcal, a menu such as "chicken and tomato soup, salad, and brown rice" can be generated. The generated menu is sent from the server to the user's device.

[0594] The device generates a list of required ingredients (e.g., "300g chicken, 2 tomatoes, salad vegetables, 2 cups brown rice") based on the received menu and displays it to the user. When the user checks and edits the list of ingredients and taps the "Order" button, the device sends the order data using the API of the online sales service. For example, a general REST API can be used here. The online sales service receives the order and sets the delivery date and time and collection method.

[0595] Once the order details are confirmed, the server generates a QR code based on that information and sends it to the user's device. The user then uses this QR code to confirm receipt of the item through the online sales service. QR codes can be generated using the zxing library, for example.

[0596] The user enters the received ingredient information into the device, which then sends the information back to the server. The server stores the received data and retrieves specific cooking instructions and video links via the cooking assistant app's API (e.g., a cooking recipe service). The retrieved information is then sent from the server to the user's device and displayed.

[0597] Furthermore, the server analyzes and records leftover ingredient information based on the user's purchase and cooking history. By incorporating this information into the next menu generation, it is possible to create recipes that minimize food waste. For example, it can suggest new salad recipes using unused tomatoes.

[0598] Examples and prompts

[0599] As a concrete example, let's consider the case of a female user in her 30s using this system for the first time. First, the user enters her profile (e.g., female in her 30s, no allergies, calorie intake goal 1500 kcal) and household ingredient inventory (e.g., rice, chicken, tomatoes) into her device. This information is saved on the server, and the generative AI model uses it to generate a menu of "chicken and tomato soup, salad, and brown rice" and sends it to the user's device. The user checks and edits the ingredient list and submits the order to the online sales service, where the order details are sent as a QR code. The user then re-enters the received ingredient information, and a link to a cooking video providing detailed cooking instructions is displayed on the user's device.

[0600] An example prompt is:

[0601] "Generate a healthy meal plan for a woman in her 30s with no allergies and a calorie intake goal of 1500 kcal. The ingredients in the household are rice, chicken, and tomatoes."

[0602] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0603] Step 1: User registration and initial setup

[0604] The user launches the application and enters health information such as age, gender, allergy information, and calorie intake goal into the device. The entered information is sent by the device to the server via an HTTP request. The server stores the received health information in a database (e.g., MySQL). Next, the user enters the food inventory in the home (e.g., rice, chicken, vegetables, etc.), and the device similarly sends this to the server and stores it in the database.

[0605] Input: age, gender, allergy information, calorie intake goal, food inventory information.

[0606] Output: Health information and food inventory information stored in a database.

[0607] Specific operation: The user enters information into the input field of the smartphone app and taps the "Save" button. This operation causes the device to send the information to the server. The server receives the data and stores it in a database.

[0608] Step 2: Meal menu generation

[0609] The server retrieves the stored health information and food inventory information from the database and sends a prompt to a generative AI model (e.g., OpenAI's GPT-4). The generative AI model generates a meal menu (e.g., "chicken and tomato soup, salad, and brown rice") and sends it to the server. The server stores the received menu in a database and sends it to the user's device.

[0610] Input: Health information, food inventory information, prompt text.

[0611] Output: The generated meal menu.

[0612] Specific operation: The server creates a prompt and sends it to the AI ​​model. The AI ​​model generates a menu and returns it to the server. The server saves the menu and sends it to the device. The user's device displays the menu via a push notification.

[0613] Step 3: Generate materials list and order

[0614] Based on the menu received, the terminal generates a list of required ingredients and displays it to the user. The user checks the list of ingredients, makes any necessary edits (e.g., changes to quantities, adding or deleting items), and confirms the "order." The terminal then sends the order data via the API of the online sales service. The online sales service receives the order data and sets the delivery date and time, etc.

[0615] Input: The generated meal menu.

[0616] Output: Generated material list, order details.

[0617] Specific operation: The terminal analyzes the menu and generates an ingredient list. The user checks and edits the list. After confirming the order, the terminal sends the order to the online sales service's API. The online sales service receives and processes the order.

[0618] Step 4: Generate and distribute QR codes

[0619] Once the order details are confirmed, the server generates a QR code based on the order information and sends it to the user's device. The user then uses this QR code when picking up the item from the online shopping service.

[0620] Input: Order details.

[0621] Output: QR code.

[0622] Specific operation: The server analyzes the order details and generates a QR code (e.g., using the zxing library). The generated QR code URL is sent to the user's device. The user displays the QR code on their device and uses it to receive the item.

[0623] Step 5: Save purchased ingredients and provide cooking instructions

[0624] The user enters the received ingredient information into the device and sends it to the server. The server stores the received information in a database and obtains specific cooking instructions and video links via the cooking assistant app's API (e.g., a cooking recipe service). The server sends this information to the user's device, which then displays it to the user.

[0625] Input: Received ingredient information.

[0626] Output: Specific cooking instructions and video links.

[0627] Specific operation: The user enters the ingredients they have purchased into their device and taps the "Save" button. The device sends the information to the server. The server saves it in a database, then calls the cooking app's API to retrieve the cooking information. The retrieved cooking instructions are sent to the user's device, which displays them to the user.

[0628] Step 6: Manage leftover ingredients and create your next recipe

[0629] The server analyzes and records leftover ingredient information based on the purchase and cooking history sent by the user. This information is reflected in the next menu generation and suggests recipes that minimize waste.

[0630] Input: purchase history, cooking history.

[0631] Output: Information on leftover ingredients, next recipe.

[0632] Specific operation: The server retrieves historical information from the database and uses the AI ​​model to calculate the remaining amount of unused ingredients. The calculation result is included in the next prompt and sent to the AI ​​model. The model generates a new menu, which the server saves in the database. The generated menu is then notified to the user's device.

[0633] (Application example 1)

[0634] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0635] In modern society, personal health management and efficient dietary habits are important issues. In particular, the lack of a system that centrally manages multiple elements, such as generating efficient meal menus based on health information and food inventory information, online ordering, and providing cooking instructions, is a problem. There is also a need for a method to provide menus optimized for each individual's health status and use ingredients without waste based on such menus.

[0636] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0637] In this invention, the server includes means for inputting and saving personal information including health information, means for inputting and saving information on household food inventory, means for generating a meal menu using the saved personal information and food ingredient information, means for displaying a list of necessary ingredients based on the generated meal menu, means for ordering ingredients via an online supermarket API, means for converting the order content into a QR code and distributing it, means for saving information on purchased ingredients and providing cooking instructions, means for recording information on leftover ingredients and reflecting it in the next menu generation, means for providing instructions to a smartphone app based on the generated meal menu and food ingredient information, means for generating prompts, and means for generating an optimal meal menu using a generative AI model. This enables the provision of individually optimized healthy menus, the use of ingredients without waste, and centralized management of online orders.

[0638] "Health information" refers to information related to an individual's health, such as an individual's age, gender, allergy information, and calorie intake goal.

[0639] "Personal information" is information necessary to identify a specific individual, including health information.

[0640] "Food inventory information" refers to information about the types and amounts of food ingredients currently stored in the home.

[0641] "Meal Menu" refers to a combination of dishes or a menu suggested based on personal health information and food inventory information.

[0642] "Ingredients list" refers to a list of the types and quantities of ingredients needed to prepare a particular meal.

[0643] An "online supermarket" is an online retail store where you can order and purchase food and daily necessities via the Internet.

[0644] "API" is an abbreviation for Application Program Interface, and refers to an interface for linking functions between software.

[0645] "QR code" refers to a two-dimensional barcode that allows for easy electronic acquisition of information.

[0646] "Cooking instructions" refers to information such as steps or tips for making a particular dish, or a video link showing how to do so.

[0647] A "prompt" is a message or instruction that requests some input or response from the user.

[0648] A "generative AI model" refers to an artificial intelligence model that generates optimal output based on input data.

[0649] "Smartphone app" refers to a software application that runs on a smartphone.

[0650] As a form for implementing this invention, we will specifically explain a series of systems that generate meal menus based on health information and ingredient inventory information, procure ingredients through collaboration with an online supermarket, and provide cooking methods.

[0651] User registration and initial settings

[0652] When a user uses the system for the first time, they first enter their profile information (age, gender, allergy information, calorie intake goal, etc.) into the device. The device sends the entered information to the server, which stores it. Next, the user enters information about food ingredients in their home (e.g., rice, chicken, vegetables, etc.) into the device, which also sends this information to the server and stores it.

[0653] Meal Menu Generation

[0654] The server retrieves the stored personal information and food inventory information and uses the generative AI model to generate several days' worth of meal menus. For example, it generates a menu of "chicken and tomato soup, salad, and brown rice" based on a calorie intake target of 1500 kcal. The generated menu is sent from the server to the user's device.

[0655] Generate and order materials lists

[0656] Based on the received menu, the terminal displays a list of required ingredients (e.g., "300g chicken, 2 tomatoes, salad vegetables, 2 cups brown rice") to the user. The user can check this list and edit it as necessary (for example, change the amount, add, or delete). The confirmed ingredient list is sent as an order from the terminal via the online supermarket's API. The online supermarket receives the order and sets the delivery date, time, and collection method.

[0657] QR Code Generation and Distribution

[0658] Once the order details are confirmed, the server converts the information into a QR code and sends it to the user's device. The user can then use the QR code received on their device to confirm receipt at the online supermarket.

[0659] Save purchased ingredients information and provide cooking instructions

[0660] The user inputs the delivered ingredients into the device and sends that information to the server. The server saves the purchased ingredient information and, based on the saved data, retrieves specific cooking instructions and video links via the cooking app's API. This information is sent from the server to the user's device, and the user can cook while referring to the cooking instructions displayed on the device.

[0661] Managing leftover ingredients and creating your next recipe

[0662] The server analyzes and records leftover ingredients based on the user's purchase and cooking history. This information is reflected in the next menu generation and creates recipes to minimize waste. For example, it might suggest a new salad recipe using unused tomatoes.

[0663] Specific examples

[0664] When a female user in her 30s uses the system for the first time, she first enters her profile (e.g., female in her 30s, no allergies, calorie intake goal 1500 kcal) and her household food inventory (e.g., rice, chicken, tomatoes) into the device. Once this information is saved on the server, the generative AI model uses it to create a menu of "chicken and tomato soup, salad, brown rice" and sends it to the user.

[0665] Once the user checks and edits the ingredient list and submits the order to the online supermarket, the order details are sent as a QR code. After that, the user re-enters the received ingredient information, and a link to a cooking video providing detailed cooking instructions is displayed on the device.

[0666] Prompt Sentence Examples

[0667] User Information:

[0668] Age: 30

[0669] Gender: Female

[0670] Calorie intake goal: 1500 kcal

[0671] Allergy Information: None

[0672] Ingredients availability information:

[0673] Rice: 2 cups

[0674] Chicken: 1 piece

[0675] Tomatoes: 3

[0676] Please suggest a healthy meal plan based on the above information.

[0677] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0678] Step 1:

[0679] The user enters their profile information into the device, which then sends the information (such as age, gender, allergy information, and calorie intake goal) to the server, where it is stored in a database.

[0680] Input: Age, gender, allergy information, calorie intake goal

[0681] Output: Personal information stored in the database

[0682] Specific operation: The user enters information using a smartphone app and presses the "Save" button. The device sends the entered information as a request to the server's API, and the server saves the information in a database.

[0683] Step 2:

[0684] The user inputs information about the food inventory in their home into the terminal. The input data (e.g., rice, chicken, tomatoes, etc.) is sent from the terminal to the server, which then stores it in a database.

[0685] Input: Ingredient information such as rice, chicken, and tomatoes

[0686] Output: Ingredient inventory information stored in the database

[0687] Specific operation: The user enters the inventory and presses the "Save" button. The terminal sends the information as a request to the server's API, and the server saves the information in the database.

[0688] Step 3:

[0689] The server retrieves the stored personal information and food inventory information and generates several days' worth of meal menus using a generative AI model.

[0690] Input: Personal information, food inventory information

[0691] Output: Generated meal menus for several days

[0692] Specific operation: The server inputs personal information and food inventory information into the AI ​​model and generates the following prompt: "User information: Age: 30, Gender: Female, Calorie intake goal: 1500 kcal, Allergy information: None, Food inventory information: Rice: 2 cups, Chicken: 1, Tomatoes: 3. Based on the above information, please suggest a healthy meal menu." This is input into the generative AI model to obtain a meal menu.

[0693] Step 4:

[0694] Based on the received menu, the device displays a list of ingredients (e.g., "300g chicken, 2 tomatoes, salad greens, 2 cups brown rice") to the user, who can then review and edit the list if necessary.

[0695] Input: Generated meal menu

[0696] Output: Materials list

[0697] Specific operation: The terminal retrieves and displays the menu, and the user confirms and edits it. The terminal visualizes and displays the ingredient list.

[0698] Step 5:

[0699] The confirmed ingredients list is then sent from the device to the online supermarket via its API, which then receives the order and sets the delivery date and time and pick-up method.

[0700] Input: Edited materials list

[0701] Output: Order information and delivery date / time / pickup method settings

[0702] Specific operation: The terminal sends an order request to the online supermarket's API, the server processes the order, and the terminal displays an order confirmation.

[0703] Step 6:

[0704] Once the order details are confirmed, the server converts the information into a QR code and sends it to the user's device. The user then uses the QR code received on their device to confirm receipt at the online supermarket.

[0705] Input: Order Information

[0706] Output: QR code

[0707] Specific operation: The order information is converted using a QR code generation library, and the generated QR code is sent to the device. The user scans the QR code when receiving the order.

[0708] Step 7:

[0709] The user inputs the delivered ingredients into the device and sends the information to the server. The server saves the purchased ingredients and retrieves specific cooking instructions and video links based on the saved data via the cooking app's API.

[0710] Input: Delivered food information

[0711] Output: Cooking instructions and video links

[0712] Specific operation: The user inputs the delivered ingredients, and the device sends the information to the server. The server accesses the cooking app's API to retrieve the information and displays it on the user's device.

[0713] Step 8:

[0714] The server analyzes and records leftover ingredients based on the purchase and cooking history sent by the user. This information is reflected in the next menu generation, and recipes are created to minimize waste.

[0715] Input: purchase history, cooking history

[0716] Output: Recipe reflected in next menu generation

[0717] Specific operation: The server analyzes purchase and cooking history data, records leftover ingredients, and reflects them in the next menu generation. If necessary, it generates new prompts and inputs them into the generative AI model to obtain recipes.

[0718] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0719] As a form for implementing the present invention, we will specifically explain a system that generates meal menus based on health information and ingredient inventory information, procures ingredients through collaboration with an online supermarket, and provides cooking methods, all of which are combined with an emotion engine that recognizes the user's emotions.

[0720] User registration and initial settings

[0721] When a user uses the system for the first time, they first enter their profile information (age, gender, allergy information, calorie intake goal, etc.) into the device. The device then sends the entered information to the server, which stores it. The user then enters information about the food ingredients in their home (e.g., rice, chicken, vegetables, etc.) into the device, which then sends this information to the server and stores it. In addition, an interface that allows the emotion engine to collect the user's emotion information is also included as part of the initial setup.

[0722] Manipulating the Emotion Engine

[0723] The emotion engine continuously collects data such as the user's facial expressions, voice, and text messages to recognize their emotions, and then analyzes and transmits emotional information such as their current mood and stress level to the server.

[0724] Meal Menu Generation

[0725] The server uses an AI model to generate several days' worth of meal menus based on stored personal information, ingredient inventory information, and emotional information provided by the emotion engine. For example, if the server detects that the user is feeling stressed, it generates a menu using ingredients that have a relaxing effect. The generated menu is then sent from the server to the user's device.

[0726] Generate and order materials lists

[0727] Based on the received menu, the terminal displays a list of required ingredients (e.g., "300g chicken, 2 tomatoes, salad vegetables, 2 cups brown rice") to the user. The user can check this list and edit it as necessary (e.g., change the amount, add, or delete). The confirmed ingredient list is sent as an order from the terminal via the online supermarket's API. The online supermarket receives the order and sets the delivery date and time and collection method.

[0728] QR Code Generation and Distribution

[0729] Once the order details are confirmed, the server converts the information into a QR code and sends it to the user's device. The user can then use the QR code received on their device to confirm receipt at the online supermarket.

[0730] Save purchased ingredients information and provide cooking instructions

[0731] The user inputs the delivered ingredients into the device and sends that information to the server. The server saves the purchased ingredient information and, based on the saved data, retrieves specific cooking instructions and video links via the cooking app's API. This information is sent from the server to the user's device, and the user can cook while referring to the cooking instructions displayed on the device.

[0732] Managing leftover ingredients and creating your next recipe

[0733] The server analyzes and records leftover ingredient information based on the user's purchase and cooking history. Furthermore, this information, along with data from the emotion engine, is taken into account when generating the next menu, creating recipes to minimize waste. For example, it can suggest new salad recipes using unused tomatoes.

[0734] Specific examples

[0735] A female user in her 30s uses the system for the first time and enters her profile information (e.g., female in her 30s, no allergies, calorie intake goal 1500 kcal) and her household food inventory (e.g., rice, chicken, tomatoes) into the device. The emotion engine detects that the user is feeling stressed. This information is also sent to the server, and the AI ​​uses it to generate a menu of "relaxing chicken and tomato soup, salad, and brown rice" and sends it to the user.

[0736] Once the user confirms and edits the ingredient list and submits the order to the online supermarket, the order details are sent as a QR code. When the user re-enters the received ingredient information, a link to a cooking video with specific cooking instructions is displayed on the device. The system aims to simplify daily meal preparation and promote health maintenance and efficient ingredient management.

[0737] The present invention reduces the burden of meal planning and food ingredient management on the user, and further makes it possible to propose menus that match the user's mood based on emotional information, thereby enabling the user to maintain a healthy diet.

[0738] The processing flow will be explained below.

[0739] Step 1:

[0740] The user enters profile information (age, gender, allergy information, calorie intake goal, etc.) into the terminal.

[0741] Step 2:

[0742] The terminal transmits the entered profile information to the server.

[0743] Step 3:

[0744] The server stores the received profile information in a database.

[0745] Step 4:

[0746] The user inputs information about food ingredients in the home into the terminal.

[0747] Step 5:

[0748] The terminal transmits the input food stock information to the server.

[0749] Step 6:

[0750] The server stores the received food stock information in a database.

[0751] Step 7:

[0752] The emotion engine collects the user's emotional information (facial expressions, voice, text messages, etc.) and recognizes emotions.

[0753] Step 8:

[0754] The emotion engine sends the recognized emotion information to the server.

[0755] Step 9:

[0756] The server acquires the stored personal information, food stock information, and emotion information.

[0757] Step 10:

[0758] The server uses an AI model to generate a meal menu for several days that takes into account emotional information. For example, if the user is feeling stressed, it will generate a menu with a relaxing effect.

[0759] Step 11:

[0760] The server transmits the generated meal menu to the user terminal.

[0761] Step 12:

[0762] The terminal displays the meal menu received from the server to the user.

[0763] Step 13:

[0764] The user reviews the meal menu and edits the ingredient list as needed.

[0765] Step 14:

[0766] The terminal sends the confirmed list of ingredients to the online supermarket's API.

[0767] Step 15:

[0768] The online supermarket processes the order based on the received ingredient list and sets the delivery date and time and collection method.

[0769] Step 16:

[0770] The server receives the order details from the online supermarket and converts them into a QR code.

[0771] Step 17:

[0772] The server sends the generated QR code to the user's terminal.

[0773] Step 18:

[0774] The user receives the QR code on their device and uses it to receive the ingredients.

[0775] Step 19:

[0776] The user inputs the delivered food information into the terminal.

[0777] Step 20:

[0778] The terminal transmits information about the delivered ingredients to the server.

[0779] Step 21:

[0780] The server stores the received purchased ingredient information in a database.

[0781] Step 22:

[0782] Based on the saved data, the server obtains specific cooking instructions and video links via the cooking app's API.

[0783] Step 23:

[0784] The server sends the acquired cooking instructions and video links to the user's terminal.

[0785] Step 24:

[0786] The user cooks the food while referring to the cooking instructions displayed on the terminal.

[0787] Step 25:

[0788] The server analyzes the purchase history and cooking history sent by the user and records information about leftover ingredients.

[0789] Step 26:

[0790] The server takes into account the remaining ingredients and emotion information and reflects them in the next menu generation. For example, it generates a menu using unused tomatoes.

[0791] This series of processing steps streamlines the user's meal preparation and health management, while providing a balanced meal plan based on emotional information.

[0792] Example 2

[0793] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0794] Conventional meal planning systems only considered personal health information and household food inventory information, and were unable to suggest menus that took the user's emotional state into account. This made it difficult to suggest appropriate menus when the user was feeling stressed or in a particular mood, limiting their ability to provide comprehensive health management for the user. Furthermore, their management of food waste to a minimum was insufficient, leaving a need for more efficient food use.

[0795] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0796] In this invention, the server includes means for inputting and saving personal information including health information, means for inputting and saving household ingredient inventory information, means for generating a meal menu using the saved personal information and ingredient inventory information, means for displaying a list of ingredients required based on the generated meal menu, means for ordering ingredients via an API of an online sales service, means for converting the order details into a QR code and distributing it, means for saving information on purchased ingredients and providing cooking instructions, means for recording information on leftover ingredients and reflecting it in generating the next menu, and means for collecting, analyzing information on the user's emotions and reflecting it in generating the meal menu. This makes it possible to propose an appropriate meal menu based on the user's health condition and emotional state, and to reduce waste by using ingredients efficiently.

[0797] "Health information" is data about the user's physical and health status, such as age, gender, allergy information, and calorie intake goals.

[0798] "Personal Information" is any data about a user, including profile information and health information.

[0799] "Home food ingredient inventory information" is information about the types and quantities of ingredients currently stored in the user's home.

[0800] A "meal menu" is a list of meals that the user plans to consume, generated based on the user's personal information and home food inventory information.

[0801] The "ingredient list" is a list of ingredients required based on the generated meal menu.

[0802] An "API for online sales services" is a program interface for ordering products online.

[0803] A "QR code" is a two-dimensional code that visually conveys information about an order.

[0804] "Purchased ingredient information" is information about ingredients purchased by a user via an online sales service.

[0805] "Cooking instructions" is information about the steps and techniques for cooking using purchased ingredients.

[0806] "Leftover ingredient information" is information about ingredients that were left over from the previous cooking session and were not used.

[0807] "Emotion information" is data relating to the user's emotional state obtained from facial expressions, voice, text, etc.

[0808] The present invention combines a series of systems that generate meal menus based on a user's health information and household food inventory information, procure ingredients in cooperation with online sales services, and provide cooking methods with an emotion engine that recognizes the user's emotion information. Specific embodiments of this system are described below.

[0809] First, when using the device for the first time, the user enters their profile information (age, gender, allergy information, calorie intake goal, etc.) into the device, which then sends and stores this information on the server. Next, the user enters information about the food ingredients in their home (rice, chicken, tomatoes, etc.) into the device, which then sends this information to the server and stores it. The initial settings for the emotion engine are also performed on the device, and an interface is built to collect the user's emotion information.

[0810] The emotion engine collects emotional data from users through facial expression analysis, voice recognition, and text analysis, and analyzes it in real time. The analysis results are sent to a server to identify their state of stress and other factors.

[0811] The server combines the stored personal information, food inventory information, and emotional information provided by the emotion engine to generate a meal menu for several days using an AI model. For example, if the server determines that the user is feeling stressed, it will suggest a menu using ingredients that have a relaxing effect. This generated menu is then sent from the server to the user's device.

[0812] The user's device creates a list of ingredients based on the received menu and displays it to the user. Once the user confirms and edits the ingredients list, the confirmed list is sent to the online supermarket via its API. The online supermarket receives the order and sets the delivery date and time and collection method.

[0813] Once the order details are confirmed, the server converts the information into a QR code and sends it to the user's device. The user uses the QR code on their device to confirm receipt at the online supermarket. Information about the delivered ingredients is again entered into the device and sent to the server. Based on this information, the server obtains specific cooking instructions and a link to a cooking video via the cooking assistance application's API and sends it to the user's device. The user cooks while referring to this information.

[0814] The server also analyzes the user's purchase and cooking history and records any leftover ingredients. The next time the server generates a menu, it takes this information and emotional information into account to create recipes that minimize waste. For example, it might suggest a new salad recipe using unused tomatoes.

[0815] The present invention makes it possible to propose an appropriate meal menu according to the user's health condition and emotional state, and also reduces waste by using ingredients more efficiently.

[0816] Specific examples

[0817] A female user in her 30s uses the system for the first time and enters her profile information (e.g., female in her 30s, no allergies, calorie intake goal 1500 kcal) and her household food inventory (e.g., rice, chicken, tomatoes) into the device. The emotion engine detects that the user is feeling stressed. This information is also sent to the server, and the AI ​​uses it to generate a menu of "relaxing chicken and tomato soup, salad, and brown rice" and sends it to the user.

[0818] Prompt Sentence Examples

[0819] "A woman in her 30s with no allergies and a calorie intake goal of 1500 kcal. She has rice, chicken, and tomatoes at home. Please suggest a meal menu that will help the user relax when they are feeling stressed."

[0820] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0821] Step 1: User registration and initial setup

[0822] When a user uses the device for the first time, they enter their profile information (e.g., age, gender, allergy information, calorie intake goal, etc.). The entered information is sent from the device to the server, which stores it in a database. Next, the user enters information about the food inventory in their home (e.g., 1 kg of rice, 500 g of chicken, 3 tomatoes) into the device, which also sends this information to the server and stores it. The device then initializes the emotion engine and builds an interface to collect the user's emotion information.

[0823] input:

[0824] Profile information (age, gender, allergy information, calorie intake goal, etc.)

[0825] Home food inventory information (rice, chicken, tomatoes, etc.)

[0826] output:

[0827] Personal information and food inventory information stored on the server

[0828] The initial settings of the emotion engine are complete

[0829] Specific behavior:

[0830] The terminal receives input information from the user and sends it to the server via the API.

[0831] The server stores the received data in a database.

[0832] The terminal displays an emotion engine setting wizard and asks the user for permission to collect emotion data.

[0833] Step 2: Manipulating the Emotion Engine

[0834] The emotion engine collects emotion data through facial expression analysis, voice recognition, and text analysis. This data is analyzed in real time to extract the user's current emotion information (e.g., stress level) and send it to the server.

[0835] input:

[0836] User facial expressions, voice, and text data

[0837] output:

[0838] Analyzed emotional information (e.g., whether the user is feeling stressed)

[0839] Specific behavior:

[0840] The emotion engine captures the user's facial expressions and voice via the camera and microphone and analyzes text messages.

[0841] The emotion engine uses machine learning algorithms to analyze the emotion data and send the results to a server.

[0842] The server integrates the received emotion information into a user profile and stores it.

[0843] Step 3: Meal menu generation

[0844] The server uses an AI model to generate a meal menu based on stored personal information, food inventory information, and emotional information. For example, if the server detects that the user is feeling stressed, it will generate a menu using ingredients that have a relaxing effect. The generated menu is then sent from the server to the user's device.

[0845] input:

[0846] personal information

[0847] Food inventory information

[0848] emotional information

[0849] output:

[0850] Generated meal menu (e.g., relaxing chicken and tomato soup, salad, and brown rice)

[0851] Specific behavior:

[0852] The server inputs the stored data into an AI model, which then generates a meal menu.

[0853] The server sends the generated menu to the terminal.

[0854] Step 4: Generate materials list and order

[0855] The device creates a list of required ingredients based on the received menu and displays it to the user. The user can check this list and change the quantities or add or delete items. The confirmed list is then sent from the device via the API of the online sales service. The online sales service receives the order details and sets the delivery date and time and collection method.

[0856] input:

[0857] Generated meal menu

[0858] output:

[0859] Required Materials List

[0860] Submitted order information

[0861] Specific behavior:

[0862] The device analyzes the meal menu and automatically generates a list of necessary ingredients.

[0863] The user edits this list, and after reviewing it, sends it to the online sales service via API.

[0864] The online sales service receives the order information and begins processing it.

[0865] Step 5: Generate and distribute QR codes

[0866] The server converts the order details into a QR code and sends it to the user's device, which the user can use to confirm receipt of the order through the online sales service.

[0867] input:

[0868] Order details

[0869] output:

[0870] QR code

[0871] Specific behavior:

[0872] The server encodes the order details into a QR code and sends the generated QR code to the terminal.

[0873] The user displays the QR code on their device and scans it upon receipt.

[0874] Step 6: Save purchased ingredients and provide cooking instructions

[0875] The user inputs the delivered ingredients into the device and sends that information to the server. The server saves the purchased ingredient information and, based on the saved data, retrieves specific cooking instructions and video links via the cooking assistance application's API. This information is sent from the server to the user's device, and the user cooks while referring to the cooking instructions displayed on the device.

[0876] input:

[0877] Purchased food information

[0878] output:

[0879] Saved ingredient information

[0880] Cooking methods and cooking video links

[0881] Specific behavior:

[0882] The user enters the received ingredient information into the terminal and sends it to the server via the API.

[0883] The server uses the stored ingredient information to connect to the API of the cooking assistance application and obtain cooking instructions and video links.

[0884] The acquired information is sent to the terminal so that the user can view and implement it.

[0885] Step 7: Manage leftover ingredients and create your next recipe

[0886] The server analyzes and records leftover ingredient information based on the purchase and cooking history sent by the user. Taking this information and data from the emotion engine into consideration, the server generates recipes that minimize waste and are reflected in the next menu creation.

[0887] input:

[0888] Purchase history

[0889] Cooking history

[0890] emotional information

[0891] output:

[0892] Leftover food information

[0893] Menu for next time

[0894] Specific behavior:

[0895] The server analyzes the data sent by the user and identifies any leftover ingredients.

[0896] The remaining ingredients will be taken into consideration when creating the next recipe, and a menu will be created to minimize waste.

[0897] The new menu is sent to the user terminal.

[0898] (Application example 2)

[0899] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0900] In modern life, it is extremely difficult to efficiently manage daily meal preparation while maintaining a healthy diet. In particular, there is a demand for home food ingredient management and flexible menu suggestions that respond to the user's mood. Furthermore, while meeting these demands, it is also necessary to achieve smooth food procurement through integration with online supermarkets. To solve these issues, it is essential to develop a system that can centrally manage the entire process and is easy for users to use.

[0901] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0902] In this invention, the server includes means for inputting and saving personal information including health information, means for inputting and saving information about home food ingredient inventory, means for generating a meal menu using the saved personal information and ingredient information, means for displaying a list of ingredients needed based on the generated meal menu, means for ordering ingredients via an online supermarket API, means for converting the order details into a QR code and distributing it, means for saving information about purchased ingredients and providing cooking instructions, means for recording information about remaining ingredients and reflecting this in generating the next menu, means for recognizing and collecting user emotional information, means for analyzing the emotional information and optimizing the meal menu based on the analysis results, means for acquiring the emotional information on a smartphone, and means for displaying the suggested meal menu via an application used in the virtual store. This allows for the suggestion of an optimal meal menu based on the user's health condition and emotions, allowing for smooth procurement of ingredients and efficient cooking based on the menu.

[0903] "Health information" refers to information related to the user's health condition and nutritional balance, such as age, gender, allergy information, and calorie intake goal.

[0904] "Personal Information" refers to information that is individually associated with a user, specifically the user's profile information, health information, and emotional information.

[0905] "Inventory information for ingredients" refers to information indicating the types and quantities of ingredients currently available in the home, such as rice, chicken, and vegetables.

[0906] "Emotional information" refers to information that indicates the user's current emotional state, such as stress level and mood.

[0907] "Virtual store" refers to a platform that provides a virtual store experience via the Internet, where users can purchase products.

[0908] "Emotion engine" refers to the algorithms and mechanisms for recognizing and collecting a user's emotional state by analyzing data such as facial expressions, voice, and text messages.

[0909] A "generative AI model" refers to an artificial intelligence algorithm that provides a rational output based on input data. Examples include GPT-4 and BERT.

[0910] A "prompt sentence" refers to the input sentence that is fed into a generative AI model, which then generates an output based on that sentence.

[0911] "Diet Menu" refers to a list of meals that are recommended for the user to consume, generated based on health and emotional information.

[0912] "Ingredient list" refers to a list of ingredients needed to cook a meal based on the menu. For example, "300g chicken, 2 tomatoes."

[0913] "Net supermarket" refers to a retail service that allows users to order food and daily necessities online. Users can purchase the products they need via the Internet.

[0914] An "API" is an interface that allows different software systems to share functionality. Specifically, you submit an order through the online supermarket's API.

[0915] A "QR code" is a two-dimensional barcode that visually encodes digital information and can be scanned with a smartphone or other device.

[0916] "Cooking method" refers to the steps for making a dish using specified ingredients. Specifically, it can be a recipe or a link to a cooking video.

[0917] A "cooking app" refers to an application that provides cooking methods and recipes that users can refer to to prepare a dish.

[0918] User registration and initial settings

[0919] When a user uses the system for the first time, they enter their profile information (age, gender, allergy information, calorie intake goal, etc.) into the device using their smartphone, and this information is sent to the server. Next, they enter information about their household food inventory (e.g., rice, chicken, vegetables, etc.) into the device, and this information is also sent and saved to the server. An interface that allows the emotion engine to collect the user's emotion information is also included as part of the initial setup.

[0920] Manipulating the Emotion Engine

[0921] The emotion engine uses the smartphone's camera and microphone to continuously collect the user's facial expressions and voice to recognize their emotions. This information is then analyzed to determine their current mood and stress level, and the results are sent to the server. The user's emotional data includes stress, relaxation, happiness, etc.

[0922] Generate a meal menu

[0923] The server generates several days' worth of meal menus using a generative AI model (e.g., GPT-4, BERT) based on stored personal information, ingredient inventory information, and emotional information provided by the emotion engine. For example, if the server recognizes that the user is feeling stressed, it generates a menu using ingredients that have a relaxing effect. The generated menu is sent from the server to the user's device.

[0924] Generate and order materials lists

[0925] The terminal displays a list of ingredients required based on the received menu (e.g., "300g chicken, 2 tomatoes, salad vegetables, 2 cups brown rice") to the user. The user can check and edit this list, and an order is placed using the online supermarket's API. The online supermarket receives the order and delivers it at the specified date and time.

[0926] QR Code Generation and Distribution

[0927] Once the order details are confirmed, the server converts the information into a QR code and sends it to the user's device. The user then uses the QR code received on their device to confirm receipt at the online supermarket.

[0928] Save purchased ingredients information and provide cooking instructions

[0929] The user inputs the delivered ingredients into the device and sends that information to the server. The server saves the purchased ingredient information and obtains specific cooking instructions and cooking video links via the cooking app's API. This information is sent from the server to the user's device, and the user can cook while referring to the cooking instructions displayed on the device.

[0930] Managing leftover ingredients and creating your next recipe

[0931] The server analyzes and records leftover ingredient information based on the purchase and cooking history sent by the user. Taking this information and data from the emotion engine into consideration, the server generates recipes to minimize waste and incorporates it into the next menu creation. For example, it can suggest new salad recipes using unused tomatoes.

[0932] Specific examples

[0933] A female user in her 30s uses the system for the first time and enters her profile information (e.g., female in her 30s, no allergies, calorie intake goal 1500 kcal) and her household food inventory (e.g., rice, chicken, tomatoes) into her device. The emotion engine detects that the user is feeling stressed, and this information is also sent to the server. Based on this, the generative AI model generates a menu of "relaxing chicken and tomato soup, salad, and brown rice" and sends it to the user. When the user checks and edits the ingredient list and submits the order to the online supermarket, the order details are sent as a QR code. When the received ingredient information is entered again, a link to a cooking video providing specific cooking instructions is displayed on the device.

[0934] Prompt Sentence Examples

[0935] For example, if the emotion engine detects that the user is feeling "stressed," the prompt for the AI ​​might look like this:

[0936] Prompt text:"

[0937] Generate a relaxing meal plan for a 30-year-old female with the following food inventory: rice, chicken, and tomatoes. The user is currently feeling stressed."

[0938] In this way, the application supports a healthy lifestyle by providing optimal meal menus tailored to the user's health condition and mood. This system simplifies daily meal preparation, enabling users to lead an efficient and healthy diet.

[0939] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0940] Step 1:

[0941] User registration and initial setup:

[0942] A user uses a smartphone to input profile information (age, gender, allergy information, calorie intake goal, etc.) into the device. The device sends this information to the server, which stores the personal information in a database. The user also enters information about the household's food inventory (e.g., rice, chicken, vegetables, etc.) into the device, which also sends and stores this information in the server. The device then sets up an interface for the emotion engine, enabling it to collect the user's emotional information. At this point, the input is the user's profile information and food inventory information, and the output is the information stored in the server's database.

[0943] Step 2:

[0944] Emotion Recognition with Emotion Engine:

[0945] The emotion engine collects the user's facial expressions and voice in real time from the smartphone's camera and microphone and analyzes the emotional data. This allows the emotion engine to recognize emotional information such as the user's stress level and mood, and sends the analysis results to the server. The input to this step is data from the camera and microphone, and the output is analyzed emotional information.

[0946] Step 3:

[0947] Generate a meal menu:

[0948] The server uses a generative AI model to generate several days' worth of meal menus based on the stored personal information, ingredient inventory information, and emotional information. Specifically, the server supplies "prompt sentences" to the generative AI model to generate menus tailored to the user's emotions. The inputs for this step are personal information, ingredient inventory information, and emotional information, and the output is the generated meal menu.

[0949] Step 4:

[0950] Generate and display a materials list:

[0951] The server generates a list of ingredients based on the generated meal menu and sends it to the terminal. The terminal displays the ingredient list to the user, who can review and edit it. The input of this step is the generated meal menu, and the output is the displayed ingredient list.

[0952] Step 5:

[0953] Collaboration with online supermarkets:

[0954] The terminal sends the ingredient list that the user has confirmed and edited to the server, and the server automatically places the order via the online supermarket's API. Once the order details are confirmed, the server converts the order details into a QR code and sends it to the terminal. The input for this step is the confirmed and edited ingredient list, and the output is the QR-coded order information.

[0955] Step 6:

[0956] Receiving and storing materials:

[0957] The user inputs the delivered ingredients into the terminal and sends it to the server. The server stores this purchased ingredient information in a database. The input of this step is the purchased ingredient information entered by the user, and the output is the information stored in the database.

[0958] Step 7:

[0959] Cooking instructions provided:

[0960] The server obtains specific cooking instructions and video links based on the purchased ingredient information via the cooking app's API and sends them to the user's device. The device then displays the cooking instructions to the user. The input for this step is the purchased ingredient information, and the output is the provided cooking instructions and video links.

[0961] Step 8:

[0962] Manage leftover ingredients and create your next recipe:

[0963] The server analyzes and records leftover ingredient information based on the purchase history and cooking history sent by the user. Based on this, it generates recipes to minimize waste when generating the next menu. The input for this step is the purchase history and cooking history, and the output is the recipe reflected in the next menu generation.

[0964] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0965] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0966] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0967] [Third embodiment]

[0968] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0969] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0970] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0971] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0972] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0973] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0974] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0975] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0976] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0977] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0978] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0979] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0980] As a form for implementing the present invention, we will specifically explain a series of systems that generate meal menus based on health information and ingredient inventory information, procure ingredients through collaboration with an online supermarket, and provide cooking methods.

[0981] User registration and initial settings

[0982] When a user uses the system for the first time, they first enter their profile information (age, gender, allergy information, calorie intake goal, etc.) into the device. The device sends the entered information to the server, which stores it. Next, the user enters information about food ingredients in their home (e.g., rice, chicken, vegetables, etc.) into the device, which also sends this information to the server and stores it.

[0983] Meal Menu Generation

[0984] The server retrieves the stored personal information and food inventory information and uses the AI ​​model to generate a meal menu for several days. For example, it generates a menu of "chicken and tomato soup, salad, and brown rice" based on a calorie intake target of 1500 kcal. The generated menu is sent from the server to the user's device.

[0985] Generate and order materials lists

[0986] Based on the received menu, the terminal displays a list of required ingredients (e.g., "300g chicken, 2 tomatoes, salad vegetables, 2 cups brown rice") to the user. The user can check this list and edit it as necessary (e.g., change the amount, add, or delete). The confirmed ingredient list is sent as an order from the terminal via the online supermarket's API. The online supermarket receives the order and sets the delivery date and time and collection method.

[0987] QR Code Generation and Distribution

[0988] Once the order details are confirmed, the server converts the information into a QR code and sends it to the user's device. The user can then use the QR code received on their device to confirm receipt at the online supermarket.

[0989] Save purchased ingredients information and provide cooking instructions

[0990] The user inputs the delivered ingredients into the device and sends that information to the server. The server saves the purchased ingredient information and, based on the saved data, retrieves specific cooking instructions and video links via the cooking app's API. This information is sent from the server to the user's device, and the user can cook while referring to the cooking instructions displayed on the device.

[0991] Managing leftover ingredients and creating your next recipe

[0992] The server analyzes and records leftover ingredient information based on the user's purchase and cooking history. This information is reflected in the next menu generation and creates recipes to minimize waste. For example, it might suggest a new salad recipe using unused tomatoes.

[0993] Specific examples

[0994] When a female user in her 30s uses the system for the first time, she first enters her profile (e.g., female in her 30s, no allergies, calorie intake goal 1500 kcal) and her household food inventory (e.g., rice, chicken, tomatoes) into the device. Once this information is saved on the server, the AI ​​generates a menu of "chicken and tomato soup, salad, brown rice" based on that information and sends it to the user.

[0995] Once the user confirms and edits the ingredient list and submits the order to the online supermarket, the order is converted into a QR code and sent to them. After that, the user re-enters the received ingredient information, and a link to a cooking video with detailed cooking instructions is displayed on the device. The system aims to simplify daily meal preparation and promote health maintenance and efficient ingredient management.

[0996] The present invention reduces the burden of meal planning and food ingredient management on the user, enabling them to maintain a healthy diet.

[0997] The processing flow will be explained below.

[0998] Step 1:

[0999] The user enters profile information (age, gender, allergy information, calorie intake goal, etc.) into the terminal.

[1000] Step 2:

[1001] The terminal transmits the entered profile information to the server.

[1002] Step 3:

[1003] The server stores the received profile information in a database.

[1004] Step 4:

[1005] The user inputs information about food ingredients in the home into the terminal.

[1006] Step 5:

[1007] The terminal transmits the input food stock information to the server.

[1008] Step 6:

[1009] The server stores the received food stock information in a database.

[1010] Step 7:

[1011] The server uses an AI model to generate a meal menu for several days based on the saved profile information and food inventory information.

[1012] Step 8:

[1013] The server transmits the generated meal menu to the user terminal.

[1014] Step 9:

[1015] The terminal displays the meal menu received from the server to the user.

[1016] Step 10:

[1017] The user reviews the meal menu and edits the ingredient list as needed.

[1018] Step 11:

[1019] The terminal sends the confirmed list of ingredients to the online supermarket's API.

[1020] Step 12:

[1021] The online supermarket processes the order based on the received ingredient list and sets the delivery date and time and collection method.

[1022] Step 13:

[1023] The server receives the order details from the online supermarket and converts them into a QR code.

[1024] Step 14:

[1025] The server sends the generated QR code to the user's terminal.

[1026] Step 15:

[1027] The user receives the QR code on their device and uses it to receive the ingredients.

[1028] Step 16:

[1029] The user inputs the delivered food information into the terminal.

[1030] Step 17:

[1031] The terminal transmits information about the delivered ingredients to the server.

[1032] Step 18:

[1033] The server stores the received purchased ingredient information in a database.

[1034] Step 19:

[1035] The server connects to the cooking app's API based on the stored ingredient information to obtain specific cooking instructions and video links.

[1036] Step 20:

[1037] The server sends the acquired cooking instructions and video links to the user's terminal.

[1038] Step 21:

[1039] The user cooks the food while referring to the cooking instructions displayed on the terminal.

[1040] Step 22:

[1041] The server analyzes and records information about leftover ingredients based on the purchase history and cooking history sent by the user.

[1042] Step 23:

[1043] The server will take the remaining ingredient information into consideration when generating a new menu next time.

[1044] This series of processing steps streamlines meal preparation and health management for users.

[1045] Example 1

[1046] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1047] Conventional meal planning systems require users to plan their own meal menus, source the ingredients, and search for cooking methods, which is time-consuming. They also tend to waste ingredients and make efficient food management difficult. There is a need for a system that can solve these problems and allow users to easily maintain healthy meals.

[1048] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1049] In this invention, the server includes means for inputting and saving user information including health information, means for inputting and saving household food inventory information, means for generating a meal menu using a generative AI model based on the saved user information and food inventory information, means for displaying a list of necessary ingredients based on the generated meal menu, means for ordering ingredients via an interface of an online sales service, means for coding and distributing the order details, means for saving information on purchased ingredients and providing cooking instructions, and means for recording information on leftover ingredients and reflecting it in generating the next menu. This allows users to easily plan healthy meals and efficiently procure and manage the necessary ingredients.

[1050] "Health information" refers to health-related data such as a user's age, gender, allergy information, and calorie intake goal.

[1051] "User Information" means general data about individuals using the system, including health information.

[1052] "Food inventory information" is data on the types and quantities of food currently held in the household.

[1053] "Storage means" refers to the methods and techniques used to record and retain input information in a database.

[1054] A "generative AI model" is an algorithm or system that uses artificial intelligence technology to generate specific information from input data.

[1055] A "meal menu" is a list of specific meals that a user plans to consume.

[1056] An "ingredient list" is a list of ingredients and quantities needed to prepare a particular meal.

[1057] An "interface for online retail services" is a means of connection to an online retailer's system for ordering products.

[1058] "Encoding" refers to converting data into a format such as a barcode or QR code.

[1059] "Purchased ingredient information" is data regarding the types and quantities of ingredients actually purchased by the user.

[1060] "Means for providing cooking instructions" refers to methods or services that allow users to refer to specific cooking procedures.

[1061] "Leftover ingredient information" is data regarding the type and amount of ingredients that were not used after cooking.

[1062] As an embodiment of the present invention, we will specifically explain a system that generates meal menus based on a user's health information and home food inventory information, procures the necessary ingredients through an online sales service, and provides cooking instructions. This system is composed of a server, terminals, and users.

[1063] First, when a user uses the system for the first time, they use their device to enter their own health information (e.g., age, gender, allergy information, calorie intake goal). The device sends this information to the server using an HTTP request, and the server stores the received information in a database. Next, the user enters information about food ingredients in their home (e.g., rice, chicken, vegetables, etc.), and the device again sends this to the server for storage. The database used can be, for example, MySQL.

[1064] Next, the server uses a generative AI model (e.g., OpenAI's GPT-4) to generate a meal menu for several days based on the stored health information and food inventory information. For example, if the calorie intake goal is 1500 kcal, a menu such as "chicken and tomato soup, salad, and brown rice" can be generated. The generated menu is sent from the server to the user's device.

[1065] The device generates a list of required ingredients (e.g., "300g chicken, 2 tomatoes, salad vegetables, 2 cups brown rice") based on the received menu and displays it to the user. When the user checks and edits the list of ingredients and taps the "Order" button, the device sends the order data using the API of the online sales service. For example, a general REST API can be used here. The online sales service receives the order and sets the delivery date and time and collection method.

[1066] Once the order details are confirmed, the server generates a QR code based on that information and sends it to the user's device. The user then uses this QR code to confirm receipt of the item through the online sales service. QR codes can be generated using the zxing library, for example.

[1067] The user enters the received ingredient information into the device, which then sends the information back to the server. The server stores the received data and retrieves specific cooking instructions and video links via the cooking assistant app's API (e.g., a cooking recipe service). The retrieved information is then sent from the server to the user's device and displayed.

[1068] Furthermore, the server analyzes and records leftover ingredient information based on the user's purchase and cooking history. By incorporating this information into the next menu generation, it is possible to create recipes that minimize food waste. For example, it can suggest new salad recipes using unused tomatoes.

[1069] Examples and prompts

[1070] As a concrete example, let's consider the case of a female user in her 30s using this system for the first time. First, the user enters her profile (e.g., female in her 30s, no allergies, calorie intake goal 1500 kcal) and household ingredient inventory (e.g., rice, chicken, tomatoes) into her device. This information is saved on the server, and the generative AI model uses it to generate a menu of "chicken and tomato soup, salad, and brown rice" and sends it to the user's device. The user checks and edits the ingredient list and submits the order to the online sales service, where the order details are sent as a QR code. The user then re-enters the received ingredient information, and a link to a cooking video providing detailed cooking instructions is displayed on the user's device.

[1071] An example prompt is:

[1072] "Generate a healthy meal plan for a woman in her 30s with no allergies and a calorie intake goal of 1500 kcal. The ingredients in the household are rice, chicken, and tomatoes."

[1073] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1074] Step 1: User registration and initial setup

[1075] The user launches the application and enters health information such as age, gender, allergy information, and calorie intake goal into the device. The entered information is sent by the device to the server via an HTTP request. The server stores the received health information in a database (e.g., MySQL). Next, the user enters the food inventory in the home (e.g., rice, chicken, vegetables, etc.), and the device similarly sends this to the server and stores it in the database.

[1076] Input: age, gender, allergy information, calorie intake goal, food inventory information.

[1077] Output: Health information and food inventory information stored in a database.

[1078] Specific operation: The user enters information into the input field of the smartphone app and taps the "Save" button. This operation causes the device to send the information to the server. The server receives the data and stores it in a database.

[1079] Step 2: Meal menu generation

[1080] The server retrieves the stored health information and food inventory information from the database and sends a prompt to a generative AI model (e.g., OpenAI's GPT-4). The generative AI model generates a meal menu (e.g., "chicken and tomato soup, salad, and brown rice") and sends it to the server. The server stores the received menu in a database and sends it to the user's device.

[1081] Input: Health information, food inventory information, prompt text.

[1082] Output: The generated meal menu.

[1083] Specific operation: The server creates a prompt and sends it to the AI ​​model. The AI ​​model generates a menu and returns it to the server. The server saves the menu and sends it to the device. The user's device displays the menu via a push notification.

[1084] Step 3: Generate materials list and order

[1085] Based on the menu received, the terminal generates a list of required ingredients and displays it to the user. The user checks the list of ingredients, makes any necessary edits (e.g., changes to quantities, adding or deleting items), and confirms the "order." The terminal then sends the order data via the API of the online sales service. The online sales service receives the order data and sets the delivery date and time, etc.

[1086] Input: The generated meal menu.

[1087] Output: Generated material list, order details.

[1088] Specific operation: The terminal analyzes the menu and generates an ingredient list. The user checks and edits the list. After confirming the order, the terminal sends the order to the online sales service's API. The online sales service receives and processes the order.

[1089] Step 4: Generate and distribute QR codes

[1090] Once the order details are confirmed, the server generates a QR code based on the order information and sends it to the user's device. The user then uses this QR code when picking up the item from the online shopping service.

[1091] Input: Order details.

[1092] Output: QR code.

[1093] Specific operation: The server analyzes the order details and generates a QR code (e.g., using the zxing library). The generated QR code URL is sent to the user's device. The user displays the QR code on their device and uses it to receive the item.

[1094] Step 5: Save purchased ingredients and provide cooking instructions

[1095] The user enters the received ingredient information into the device and sends it to the server. The server stores the received information in a database and obtains specific cooking instructions and video links via the cooking assistant app's API (e.g., a cooking recipe service). The server sends this information to the user's device, which then displays it to the user.

[1096] Input: Received ingredient information.

[1097] Output: Specific cooking instructions and video links.

[1098] Specific operation: The user enters the ingredients they have purchased into their device and taps the "Save" button. The device sends the information to the server. The server saves it in a database, then calls the cooking app's API to retrieve the cooking information. The retrieved cooking instructions are sent to the user's device, which displays them to the user.

[1099] Step 6: Manage leftover ingredients and create your next recipe

[1100] The server analyzes and records leftover ingredient information based on the purchase and cooking history sent by the user. This information is reflected in the next menu generation and suggests recipes that minimize waste.

[1101] Input: purchase history, cooking history.

[1102] Output: Information on leftover ingredients, next recipe.

[1103] Specific operation: The server retrieves historical information from the database and uses the AI ​​model to calculate the remaining amount of unused ingredients. The calculation result is included in the next prompt and sent to the AI ​​model. The model generates a new menu, which the server saves in the database. The generated menu is then notified to the user's device.

[1104] (Application example 1)

[1105] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1106] In modern society, personal health management and efficient dietary habits are important issues. In particular, the lack of a system that centrally manages multiple elements, such as generating efficient meal menus based on health information and food inventory information, online ordering, and providing cooking instructions, is a problem. There is also a need for a method to provide menus optimized for each individual's health status and use ingredients without waste based on such menus.

[1107] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1108] In this invention, the server includes means for inputting and saving personal information including health information, means for inputting and saving information on household food inventory, means for generating a meal menu using the saved personal information and food ingredient information, means for displaying a list of necessary ingredients based on the generated meal menu, means for ordering ingredients via an online supermarket API, means for converting the order content into a QR code and distributing it, means for saving information on purchased ingredients and providing cooking instructions, means for recording information on leftover ingredients and reflecting it in the next menu generation, means for providing instructions to a smartphone app based on the generated meal menu and food ingredient information, means for generating prompts, and means for generating an optimal meal menu using a generative AI model. This enables the provision of individually optimized healthy menus, the use of ingredients without waste, and centralized management of online orders.

[1109] "Health information" refers to information related to an individual's health, such as an individual's age, gender, allergy information, and calorie intake goal.

[1110] "Personal information" is information necessary to identify a specific individual, including health information.

[1111] "Food inventory information" refers to information about the types and amounts of food ingredients currently stored in the home.

[1112] "Meal Menu" refers to a combination of dishes or a menu suggested based on personal health information and food inventory information.

[1113] "Ingredients list" refers to a list of the types and quantities of ingredients needed to prepare a particular meal.

[1114] An "online supermarket" is an online retail store where you can order and purchase food and daily necessities via the Internet.

[1115] "API" is an abbreviation for Application Program Interface, and refers to an interface for linking functions between software.

[1116] "QR code" refers to a two-dimensional barcode that allows for easy electronic acquisition of information.

[1117] "Cooking instructions" refers to information such as steps or tips for making a particular dish, or a video link showing how to do so.

[1118] A "prompt" is a message or instruction that requests some input or response from the user.

[1119] A "generative AI model" refers to an artificial intelligence model that generates optimal output based on input data.

[1120] "Smartphone app" refers to a software application that runs on a smartphone.

[1121] As a form for implementing this invention, we will specifically explain a series of systems that generate meal menus based on health information and ingredient inventory information, procure ingredients through collaboration with an online supermarket, and provide cooking methods.

[1122] User registration and initial settings

[1123] When a user uses the system for the first time, they first enter their profile information (age, gender, allergy information, calorie intake goal, etc.) into the device. The device sends the entered information to the server, which stores it. Next, the user enters information about food ingredients in their home (e.g., rice, chicken, vegetables, etc.) into the device, which also sends this information to the server and stores it.

[1124] Meal Menu Generation

[1125] The server retrieves the stored personal information and food inventory information and uses the generative AI model to generate several days' worth of meal menus. For example, it generates a menu of "chicken and tomato soup, salad, and brown rice" based on a calorie intake target of 1500 kcal. The generated menu is sent from the server to the user's device.

[1126] Generate and order materials lists

[1127] Based on the received menu, the terminal displays a list of required ingredients (e.g., "300g chicken, 2 tomatoes, salad vegetables, 2 cups brown rice") to the user. The user can check this list and edit it as necessary (for example, change the amount, add, or delete). The confirmed ingredient list is sent as an order from the terminal via the online supermarket's API. The online supermarket receives the order and sets the delivery date, time, and collection method.

[1128] QR Code Generation and Distribution

[1129] Once the order details are confirmed, the server converts the information into a QR code and sends it to the user's device. The user can then use the QR code received on their device to confirm receipt at the online supermarket.

[1130] Save purchased ingredients information and provide cooking instructions

[1131] The user inputs the delivered ingredients into the device and sends that information to the server. The server saves the purchased ingredient information and, based on the saved data, retrieves specific cooking instructions and video links via the cooking app's API. This information is sent from the server to the user's device, and the user can cook while referring to the cooking instructions displayed on the device.

[1132] Managing leftover ingredients and creating your next recipe

[1133] The server analyzes and records leftover ingredients based on the user's purchase and cooking history. This information is reflected in the next menu generation and creates recipes to minimize waste. For example, it might suggest a new salad recipe using unused tomatoes.

[1134] Specific examples

[1135] When a female user in her 30s uses the system for the first time, she first enters her profile (e.g., female in her 30s, no allergies, calorie intake goal 1500 kcal) and her household food inventory (e.g., rice, chicken, tomatoes) into the device. Once this information is saved on the server, the generative AI model uses it to create a menu of "chicken and tomato soup, salad, brown rice" and sends it to the user.

[1136] Once the user checks and edits the ingredient list and submits the order to the online supermarket, the order details are sent as a QR code. After that, the user re-enters the received ingredient information, and a link to a cooking video providing detailed cooking instructions is displayed on the device.

[1137] Prompt Sentence Examples

[1138] User Information:

[1139] Age: 30

[1140] Gender: Female

[1141] Calorie intake goal: 1500 kcal

[1142] Allergy Information: None

[1143] Ingredients availability information:

[1144] Rice: 2 cups

[1145] Chicken: 1 piece

[1146] Tomatoes: 3

[1147] Please suggest a healthy meal plan based on the above information.

[1148] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1149] Step 1:

[1150] The user enters their profile information into the device, which then sends the information (such as age, gender, allergy information, and calorie intake goal) to the server, where it is stored in a database.

[1151] Input: Age, gender, allergy information, calorie intake goal

[1152] Output: Personal information stored in the database

[1153] Specific operation: The user enters information using a smartphone app and presses the "Save" button. The device sends the entered information as a request to the server's API, and the server saves the information in a database.

[1154] Step 2:

[1155] The user inputs information about the food inventory in their home into the terminal. The input data (e.g., rice, chicken, tomatoes, etc.) is sent from the terminal to the server, which then stores it in a database.

[1156] Input: Ingredient information such as rice, chicken, and tomatoes

[1157] Output: Ingredient inventory information stored in the database

[1158] Specific operation: The user enters the inventory and presses the "Save" button. The terminal sends the information as a request to the server's API, and the server saves the information in the database.

[1159] Step 3:

[1160] The server retrieves the stored personal information and food inventory information and generates several days' worth of meal menus using a generative AI model.

[1161] Input: Personal information, food inventory information

[1162] Output: Generated meal menus for several days

[1163] Specific operation: The server inputs personal information and food inventory information into the AI ​​model and generates the following prompt: "User information: Age: 30, Gender: Female, Calorie intake goal: 1500 kcal, Allergy information: None, Food inventory information: Rice: 2 cups, Chicken: 1, Tomatoes: 3. Based on the above information, please suggest a healthy meal menu." This is input into the generative AI model to obtain a meal menu.

[1164] Step 4:

[1165] Based on the received menu, the device displays a list of ingredients (e.g., "300g chicken, 2 tomatoes, salad greens, 2 cups brown rice") to the user, who can then review and edit the list if necessary.

[1166] Input: Generated meal menu

[1167] Output: Materials list

[1168] Specific operation: The terminal retrieves and displays the menu, and the user confirms and edits it. The terminal visualizes and displays the ingredient list.

[1169] Step 5:

[1170] The confirmed ingredients list is then sent from the device to the online supermarket via its API, which then receives the order and sets the delivery date and time and pick-up method.

[1171] Input: Edited materials list

[1172] Output: Order information and delivery date / time / pickup method settings

[1173] Specific operation: The terminal sends an order request to the online supermarket's API, the server processes the order, and the terminal displays an order confirmation.

[1174] Step 6:

[1175] Once the order details are confirmed, the server converts the information into a QR code and sends it to the user's device. The user then uses the QR code received on their device to confirm receipt at the online supermarket.

[1176] Input: Order Information

[1177] Output: QR code

[1178] Specific operation: The order information is converted using a QR code generation library, and the generated QR code is sent to the device. The user scans the QR code when receiving the order.

[1179] Step 7:

[1180] The user inputs the delivered ingredients into the device and sends the information to the server. The server saves the purchased ingredients and retrieves specific cooking instructions and video links based on the saved data via the cooking app's API.

[1181] Input: Delivered food information

[1182] Output: Cooking instructions and video links

[1183] Specific operation: The user inputs the delivered ingredients, and the device sends the information to the server. The server accesses the cooking app's API to retrieve the information and displays it on the user's device.

[1184] Step 8:

[1185] The server analyzes and records leftover ingredients based on the purchase and cooking history sent by the user. This information is reflected in the next menu generation, and recipes are created to minimize waste.

[1186] Input: purchase history, cooking history

[1187] Output: Recipe reflected in next menu generation

[1188] Specific operation: The server analyzes purchase and cooking history data, records leftover ingredients, and reflects them in the next menu generation. If necessary, it generates new prompts and inputs them into the generative AI model to obtain recipes.

[1189] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1190] As a form for implementing the present invention, we will specifically explain a system that generates meal menus based on health information and ingredient inventory information, procures ingredients through collaboration with an online supermarket, and provides cooking methods, all of which are combined with an emotion engine that recognizes the user's emotions.

[1191] User registration and initial settings

[1192] When a user uses the system for the first time, they first enter their profile information (age, gender, allergy information, calorie intake goal, etc.) into the device. The device then sends the entered information to the server, which stores it. The user then enters information about the food ingredients in their home (e.g., rice, chicken, vegetables, etc.) into the device, which then sends this information to the server and stores it. In addition, an interface that allows the emotion engine to collect the user's emotion information is also included as part of the initial setup.

[1193] Manipulating the Emotion Engine

[1194] The emotion engine continuously collects data such as the user's facial expressions, voice, and text messages to recognize their emotions, and then analyzes and transmits emotional information such as their current mood and stress level to the server.

[1195] Meal Menu Generation

[1196] The server uses an AI model to generate several days' worth of meal menus based on stored personal information, ingredient inventory information, and emotional information provided by the emotion engine. For example, if the server detects that the user is feeling stressed, it generates a menu using ingredients that have a relaxing effect. The generated menu is then sent from the server to the user's device.

[1197] Generate and order materials lists

[1198] Based on the received menu, the terminal displays a list of required ingredients (e.g., "300g chicken, 2 tomatoes, salad vegetables, 2 cups brown rice") to the user. The user can check this list and edit it as necessary (e.g., change the amount, add, or delete). The confirmed ingredient list is sent as an order from the terminal via the online supermarket's API. The online supermarket receives the order and sets the delivery date and time and collection method.

[1199] QR Code Generation and Distribution

[1200] Once the order details are confirmed, the server converts the information into a QR code and sends it to the user's device. The user can then use the QR code received on their device to confirm receipt at the online supermarket.

[1201] Save purchased ingredients information and provide cooking instructions

[1202] The user inputs the delivered ingredients into the device and sends that information to the server. The server saves the purchased ingredient information and, based on the saved data, retrieves specific cooking instructions and video links via the cooking app's API. This information is sent from the server to the user's device, and the user can cook while referring to the cooking instructions displayed on the device.

[1203] Managing leftover ingredients and creating your next recipe

[1204] The server analyzes and records leftover ingredient information based on the user's purchase and cooking history. Furthermore, this information, along with data from the emotion engine, is taken into account when generating the next menu, creating recipes to minimize waste. For example, it can suggest new salad recipes using unused tomatoes.

[1205] Specific examples

[1206] A female user in her 30s uses the system for the first time and enters her profile information (e.g., female in her 30s, no allergies, calorie intake goal 1500 kcal) and her household food inventory (e.g., rice, chicken, tomatoes) into the device. The emotion engine detects that the user is feeling stressed. This information is also sent to the server, and the AI ​​uses it to generate a menu of "relaxing chicken and tomato soup, salad, and brown rice" and sends it to the user.

[1207] Once the user confirms and edits the ingredient list and submits the order to the online supermarket, the order details are sent as a QR code. When the user re-enters the received ingredient information, a link to a cooking video with specific cooking instructions is displayed on the device. The system aims to simplify daily meal preparation and promote health maintenance and efficient ingredient management.

[1208] The present invention reduces the burden of meal planning and food ingredient management on the user, and further makes it possible to propose menus that match the user's mood based on emotional information, thereby enabling the user to maintain a healthy diet.

[1209] The processing flow will be explained below.

[1210] Step 1:

[1211] The user enters profile information (age, gender, allergy information, calorie intake goal, etc.) into the terminal.

[1212] Step 2:

[1213] The terminal transmits the entered profile information to the server.

[1214] Step 3:

[1215] The server stores the received profile information in a database.

[1216] Step 4:

[1217] The user inputs information about food ingredients in the home into the terminal.

[1218] Step 5:

[1219] The terminal transmits the input food stock information to the server.

[1220] Step 6:

[1221] The server stores the received food stock information in a database.

[1222] Step 7:

[1223] The emotion engine collects the user's emotional information (facial expressions, voice, text messages, etc.) and recognizes emotions.

[1224] Step 8:

[1225] The emotion engine sends the recognized emotion information to the server.

[1226] Step 9:

[1227] The server acquires the stored personal information, food stock information, and emotion information.

[1228] Step 10:

[1229] The server uses an AI model to generate a meal menu for several days that takes into account emotional information. For example, if the user is feeling stressed, it will generate a menu with a relaxing effect.

[1230] Step 11:

[1231] The server transmits the generated meal menu to the user terminal.

[1232] Step 12:

[1233] The terminal displays the meal menu received from the server to the user.

[1234] Step 13:

[1235] The user reviews the meal menu and edits the ingredient list as needed.

[1236] Step 14:

[1237] The terminal sends the confirmed list of ingredients to the online supermarket's API.

[1238] Step 15:

[1239] The online supermarket processes the order based on the received ingredient list and sets the delivery date and time and collection method.

[1240] Step 16:

[1241] The server receives the order details from the online supermarket and converts them into a QR code.

[1242] Step 17:

[1243] The server sends the generated QR code to the user's terminal.

[1244] Step 18:

[1245] The user receives the QR code on their device and uses it to receive the ingredients.

[1246] Step 19:

[1247] The user inputs the delivered food information into the terminal.

[1248] Step 20:

[1249] The terminal transmits information about the delivered ingredients to the server.

[1250] Step 21:

[1251] The server stores the received purchased ingredient information in a database.

[1252] Step 22:

[1253] Based on the saved data, the server obtains specific cooking instructions and video links via the cooking app's API.

[1254] Step 23:

[1255] The server sends the acquired cooking instructions and video links to the user's terminal.

[1256] Step 24:

[1257] The user cooks the food while referring to the cooking instructions displayed on the terminal.

[1258] Step 25:

[1259] The server analyzes the purchase history and cooking history sent by the user and records information about leftover ingredients.

[1260] Step 26:

[1261] The server takes into account the remaining ingredients and emotion information and reflects them in the next menu generation. For example, it generates a menu using unused tomatoes.

[1262] This series of processing steps streamlines the user's meal preparation and health management, while providing a balanced meal plan based on emotional information.

[1263] Example 2

[1264] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1265] Conventional meal planning systems only considered personal health information and household food inventory information, and were unable to suggest menus that took the user's emotional state into account. This made it difficult to suggest appropriate menus when the user was feeling stressed or in a particular mood, limiting their ability to provide comprehensive health management for the user. Furthermore, their management of food waste to a minimum was insufficient, leaving a need for more efficient food use.

[1266] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1267] In this invention, the server includes means for inputting and saving personal information including health information, means for inputting and saving household ingredient inventory information, means for generating a meal menu using the saved personal information and ingredient inventory information, means for displaying a list of ingredients required based on the generated meal menu, means for ordering ingredients via an API of an online sales service, means for converting the order details into a QR code and distributing it, means for saving information on purchased ingredients and providing cooking instructions, means for recording information on leftover ingredients and reflecting it in generating the next menu, and means for collecting, analyzing information on the user's emotions and reflecting it in generating the meal menu. This makes it possible to propose an appropriate meal menu based on the user's health condition and emotional state, and to reduce waste by using ingredients efficiently.

[1268] "Health information" is data about the user's physical and health status, such as age, gender, allergy information, and calorie intake goals.

[1269] "Personal Information" is any data about a user, including profile information and health information.

[1270] "Home food ingredient inventory information" is information about the types and quantities of ingredients currently stored in the user's home.

[1271] A "meal menu" is a list of meals that the user plans to consume, generated based on the user's personal information and home food inventory information.

[1272] The "ingredient list" is a list of ingredients required based on the generated meal menu.

[1273] An "API for online sales services" is a program interface for ordering products online.

[1274] A "QR code" is a two-dimensional code that visually conveys information about an order.

[1275] "Purchased ingredient information" is information about ingredients purchased by a user via an online sales service.

[1276] "Cooking instructions" is information about the steps and techniques for cooking using purchased ingredients.

[1277] "Leftover ingredient information" is information about ingredients that were left over from the previous cooking session and were not used.

[1278] "Emotion information" is data relating to the user's emotional state obtained from facial expressions, voice, text, etc.

[1279] The present invention combines a series of systems that generate meal menus based on a user's health information and household food inventory information, procure ingredients in cooperation with online sales services, and provide cooking methods with an emotion engine that recognizes the user's emotion information. Specific embodiments of this system are described below.

[1280] First, when using the device for the first time, the user enters their profile information (age, gender, allergy information, calorie intake goal, etc.) into the device, which then sends and stores this information on the server. Next, the user enters information about the food ingredients in their home (rice, chicken, tomatoes, etc.) into the device, which then sends this information to the server and stores it. The initial settings for the emotion engine are also performed on the device, and an interface is built to collect the user's emotion information.

[1281] The emotion engine collects emotional data from users through facial expression analysis, voice recognition, and text analysis, and analyzes it in real time. The analysis results are sent to a server to identify their state of stress and other factors.

[1282] The server combines the stored personal information, food inventory information, and emotional information provided by the emotion engine to generate a meal menu for several days using an AI model. For example, if the server determines that the user is feeling stressed, it will suggest a menu using ingredients that have a relaxing effect. This generated menu is then sent from the server to the user's device.

[1283] The user's device creates a list of ingredients based on the received menu and displays it to the user. Once the user confirms and edits the ingredients list, the confirmed list is sent to the online supermarket via its API. The online supermarket receives the order and sets the delivery date and time and collection method.

[1284] Once the order details are confirmed, the server converts the information into a QR code and sends it to the user's device. The user uses the QR code on their device to confirm receipt at the online supermarket. Information about the delivered ingredients is again entered into the device and sent to the server. Based on this information, the server obtains specific cooking instructions and a link to a cooking video via the cooking assistance application's API and sends it to the user's device. The user cooks while referring to this information.

[1285] The server also analyzes the user's purchase and cooking history and records any leftover ingredients. The next time the server generates a menu, it takes this information and emotional information into account to create recipes that minimize waste. For example, it might suggest a new salad recipe using unused tomatoes.

[1286] The present invention makes it possible to propose an appropriate meal menu according to the user's health condition and emotional state, and also reduces waste by using ingredients more efficiently.

[1287] Specific examples

[1288] A female user in her 30s uses the system for the first time and enters her profile information (e.g., female in her 30s, no allergies, calorie intake goal 1500 kcal) and her household food inventory (e.g., rice, chicken, tomatoes) into the device. The emotion engine detects that the user is feeling stressed. This information is also sent to the server, and the AI ​​uses it to generate a menu of "relaxing chicken and tomato soup, salad, and brown rice" and sends it to the user.

[1289] Prompt Sentence Examples

[1290] "A woman in her 30s with no allergies and a calorie intake goal of 1500 kcal. She has rice, chicken, and tomatoes at home. Please suggest a meal menu that will help the user relax when they are feeling stressed."

[1291] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1292] Step 1: User registration and initial setup

[1293] When a user uses the device for the first time, they enter their profile information (e.g., age, gender, allergy information, calorie intake goal, etc.). The entered information is sent from the device to the server, which stores it in a database. Next, the user enters information about the food inventory in their home (e.g., 1 kg of rice, 500 g of chicken, 3 tomatoes) into the device, which also sends this information to the server and stores it. The device then initializes the emotion engine and builds an interface to collect the user's emotion information.

[1294] input:

[1295] Profile information (age, gender, allergy information, calorie intake goal, etc.)

[1296] Home food inventory information (rice, chicken, tomatoes, etc.)

[1297] output:

[1298] Personal information and food inventory information stored on the server

[1299] The initial settings of the emotion engine are complete

[1300] Specific behavior:

[1301] The terminal receives input information from the user and sends it to the server via the API.

[1302] The server stores the received data in a database.

[1303] The terminal displays an emotion engine setting wizard and asks the user for permission to collect emotion data.

[1304] Step 2: Manipulating the Emotion Engine

[1305] The emotion engine collects emotion data through facial expression analysis, voice recognition, and text analysis. This data is analyzed in real time to extract the user's current emotion information (e.g., stress level) and send it to the server.

[1306] input:

[1307] User facial expressions, voice, and text data

[1308] output:

[1309] Analyzed emotional information (e.g., whether the user is feeling stressed)

[1310] Specific behavior:

[1311] The emotion engine captures the user's facial expressions and voice via the camera and microphone and analyzes text messages.

[1312] The emotion engine uses machine learning algorithms to analyze the emotion data and send the results to a server.

[1313] The server integrates the received emotion information into a user profile and stores it.

[1314] Step 3: Meal menu generation

[1315] The server uses an AI model to generate a meal menu based on stored personal information, food inventory information, and emotional information. For example, if the server detects that the user is feeling stressed, it will generate a menu using ingredients that have a relaxing effect. The generated menu is then sent from the server to the user's device.

[1316] input:

[1317] personal information

[1318] Food inventory information

[1319] emotional information

[1320] output:

[1321] Generated meal menu (e.g., relaxing chicken and tomato soup, salad, and brown rice)

[1322] Specific behavior:

[1323] The server inputs the stored data into an AI model, which then generates a meal menu.

[1324] The server sends the generated menu to the terminal.

[1325] Step 4: Generate materials list and order

[1326] The device creates a list of required ingredients based on the received menu and displays it to the user. The user can check this list and change the quantities or add or delete items. The confirmed list is then sent from the device via the API of the online sales service. The online sales service receives the order details and sets the delivery date and time and collection method.

[1327] input:

[1328] Generated meal menu

[1329] output:

[1330] Required Materials List

[1331] Submitted order information

[1332] Specific behavior:

[1333] The device analyzes the meal menu and automatically generates a list of necessary ingredients.

[1334] The user edits this list, and after reviewing it, sends it to the online sales service via API.

[1335] The online sales service receives the order information and begins processing it.

[1336] Step 5: Generate and distribute QR codes

[1337] The server converts the order details into a QR code and sends it to the user's device, which the user can use to confirm receipt of the order through the online sales service.

[1338] input:

[1339] Order details

[1340] output:

[1341] QR code

[1342] Specific behavior:

[1343] The server encodes the order details into a QR code and sends the generated QR code to the terminal.

[1344] The user displays the QR code on their device and scans it upon receipt.

[1345] Step 6: Save purchased ingredients and provide cooking instructions

[1346] The user inputs the delivered ingredients into the device and sends that information to the server. The server saves the purchased ingredient information and, based on the saved data, retrieves specific cooking instructions and video links via the cooking assistance application's API. This information is sent from the server to the user's device, and the user cooks while referring to the cooking instructions displayed on the device.

[1347] input:

[1348] Purchased food information

[1349] output:

[1350] Saved ingredient information

[1351] Cooking methods and cooking video links

[1352] Specific behavior:

[1353] The user enters the received ingredient information into the terminal and sends it to the server via the API.

[1354] The server uses the stored ingredient information to connect to the API of the cooking assistance application and obtain cooking instructions and video links.

[1355] The acquired information is sent to the terminal so that the user can view and implement it.

[1356] Step 7: Manage leftover ingredients and create your next recipe

[1357] The server analyzes and records leftover ingredient information based on the purchase and cooking history sent by the user. Taking this information and data from the emotion engine into consideration, the server generates recipes that minimize waste and are reflected in the next menu creation.

[1358] input:

[1359] Purchase history

[1360] Cooking history

[1361] emotional information

[1362] output:

[1363] Leftover food information

[1364] Menu for next time

[1365] Specific behavior:

[1366] The server analyzes the data sent by the user and identifies any leftover ingredients.

[1367] The remaining ingredients will be taken into consideration when creating the next recipe, and a menu will be created to minimize waste.

[1368] The new menu is sent to the user terminal.

[1369] (Application example 2)

[1370] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1371] In modern life, it is extremely difficult to efficiently manage daily meal preparation while maintaining a healthy diet. In particular, there is a demand for home food ingredient management and flexible menu suggestions that respond to the user's mood. Furthermore, while meeting these demands, it is also necessary to achieve smooth food procurement through integration with online supermarkets. To solve these issues, it is essential to develop a system that can centrally manage the entire process and is easy for users to use.

[1372] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1373] In this invention, the server includes means for inputting and saving personal information including health information, means for inputting and saving information about home food ingredient inventory, means for generating a meal menu using the saved personal information and ingredient information, means for displaying a list of ingredients needed based on the generated meal menu, means for ordering ingredients via an online supermarket API, means for converting the order details into a QR code and distributing it, means for saving information about purchased ingredients and providing cooking instructions, means for recording information about remaining ingredients and reflecting this in generating the next menu, means for recognizing and collecting user emotional information, means for analyzing the emotional information and optimizing the meal menu based on the analysis results, means for acquiring the emotional information on a smartphone, and means for displaying the suggested meal menu via an application used in the virtual store. This allows for the suggestion of an optimal meal menu based on the user's health condition and emotions, allowing for smooth procurement of ingredients and efficient cooking based on the menu.

[1374] "Health information" refers to information related to the user's health condition and nutritional balance, such as age, gender, allergy information, and calorie intake goal.

[1375] "Personal Information" refers to information that is individually associated with a user, specifically the user's profile information, health information, and emotional information.

[1376] "Inventory information for ingredients" refers to information indicating the types and quantities of ingredients currently available in the home, such as rice, chicken, and vegetables.

[1377] "Emotional information" refers to information that indicates the user's current emotional state, such as stress level and mood.

[1378] "Virtual store" refers to a platform that provides a virtual store experience via the Internet, where users can purchase products.

[1379] "Emotion engine" refers to the algorithms and mechanisms for recognizing and collecting a user's emotional state by analyzing data such as facial expressions, voice, and text messages.

[1380] A "generative AI model" refers to an artificial intelligence algorithm that provides a rational output based on input data. Examples include GPT-4 and BERT.

[1381] A "prompt sentence" refers to the input sentence that is fed into a generative AI model, which then generates an output based on that sentence.

[1382] "Diet Menu" refers to a list of meals that are recommended for the user to consume, generated based on health and emotional information.

[1383] "Ingredient list" refers to a list of ingredients needed to cook a meal based on the menu. For example, "300g chicken, 2 tomatoes."

[1384] "Net supermarket" refers to a retail service that allows users to order food and daily necessities online. Users can purchase the products they need via the Internet.

[1385] An "API" is an interface that allows different software systems to share functionality. Specifically, you submit an order through the online supermarket's API.

[1386] A "QR code" is a two-dimensional barcode that visually encodes digital information and can be scanned with a smartphone or other device.

[1387] "Cooking method" refers to the steps for making a dish using specified ingredients. Specifically, it can be a recipe or a link to a cooking video.

[1388] A "cooking app" refers to an application that provides cooking methods and recipes that users can refer to to prepare a dish.

[1389] User registration and initial settings

[1390] When a user uses the system for the first time, they enter their profile information (age, gender, allergy information, calorie intake goal, etc.) into the device using their smartphone, and this information is sent to the server. Next, they enter information about their household food inventory (e.g., rice, chicken, vegetables, etc.) into the device, and this information is also sent and saved to the server. An interface that allows the emotion engine to collect the user's emotion information is also included as part of the initial setup.

[1391] Manipulating the Emotion Engine

[1392] The emotion engine uses the smartphone's camera and microphone to continuously collect the user's facial expressions and voice to recognize their emotions. This information is then analyzed to determine their current mood and stress level, and the results are sent to the server. The user's emotional data includes stress, relaxation, happiness, etc.

[1393] Generate a meal menu

[1394] The server generates several days' worth of meal menus using a generative AI model (e.g., GPT-4, BERT) based on stored personal information, ingredient inventory information, and emotional information provided by the emotion engine. For example, if the server recognizes that the user is feeling stressed, it generates a menu using ingredients that have a relaxing effect. The generated menu is sent from the server to the user's device.

[1395] Generate and order materials lists

[1396] The terminal displays a list of ingredients required based on the received menu (e.g., "300g chicken, 2 tomatoes, salad vegetables, 2 cups brown rice") to the user. The user can check and edit this list, and an order is placed using the online supermarket's API. The online supermarket receives the order and delivers it at the specified date and time.

[1397] QR Code Generation and Distribution

[1398] Once the order details are confirmed, the server converts the information into a QR code and sends it to the user's device. The user then uses the QR code received on their device to confirm receipt at the online supermarket.

[1399] Save purchased ingredients information and provide cooking instructions

[1400] The user inputs the delivered ingredients into the device and sends that information to the server. The server saves the purchased ingredient information and obtains specific cooking instructions and cooking video links via the cooking app's API. This information is sent from the server to the user's device, and the user can cook while referring to the cooking instructions displayed on the device.

[1401] Managing leftover ingredients and creating your next recipe

[1402] The server analyzes and records leftover ingredient information based on the purchase and cooking history sent by the user. Taking this information and data from the emotion engine into consideration, the server generates recipes to minimize waste and incorporates it into the next menu creation. For example, it can suggest new salad recipes using unused tomatoes.

[1403] Specific examples

[1404] A female user in her 30s uses the system for the first time and enters her profile information (e.g., female in her 30s, no allergies, calorie intake goal 1500 kcal) and her household food inventory (e.g., rice, chicken, tomatoes) into her device. The emotion engine detects that the user is feeling stressed, and this information is also sent to the server. Based on this, the generative AI model generates a menu of "relaxing chicken and tomato soup, salad, and brown rice" and sends it to the user. When the user checks and edits the ingredient list and submits the order to the online supermarket, the order details are sent as a QR code. When the received ingredient information is entered again, a link to a cooking video providing specific cooking instructions is displayed on the device.

[1405] Prompt Sentence Examples

[1406] For example, if the emotion engine detects that the user is feeling "stressed," the prompt for the AI ​​might look like this:

[1407] Prompt text:"

[1408] Generate a relaxing meal plan for a 30-year-old female with the following food inventory: rice, chicken, and tomatoes. The user is currently feeling stressed."

[1409] In this way, the application supports a healthy lifestyle by providing optimal meal menus tailored to the user's health condition and mood. This system simplifies daily meal preparation, enabling users to lead an efficient and healthy diet.

[1410] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1411] Step 1:

[1412] User registration and initial setup:

[1413] A user uses a smartphone to input profile information (age, gender, allergy information, calorie intake goal, etc.) into the device. The device sends this information to the server, which stores the personal information in a database. The user also enters information about the household's food inventory (e.g., rice, chicken, vegetables, etc.) into the device, which also sends and stores this information in the server. The device then sets up an interface for the emotion engine, enabling it to collect the user's emotional information. At this point, the input is the user's profile information and food inventory information, and the output is the information stored in the server's database.

[1414] Step 2:

[1415] Emotion Recognition with Emotion Engine:

[1416] The emotion engine collects the user's facial expressions and voice in real time from the smartphone's camera and microphone and analyzes the emotional data. This allows the emotion engine to recognize emotional information such as the user's stress level and mood, and sends the analysis results to the server. The input to this step is data from the camera and microphone, and the output is analyzed emotional information.

[1417] Step 3:

[1418] Generate a meal menu:

[1419] The server uses a generative AI model to generate several days' worth of meal menus based on the stored personal information, ingredient inventory information, and emotional information. Specifically, the server supplies "prompt sentences" to the generative AI model to generate menus tailored to the user's emotions. The inputs for this step are personal information, ingredient inventory information, and emotional information, and the output is the generated meal menu.

[1420] Step 4:

[1421] Generate and display a materials list:

[1422] The server generates a list of ingredients based on the generated meal menu and sends it to the terminal. The terminal displays the ingredient list to the user, who can review and edit it. The input of this step is the generated meal menu, and the output is the displayed ingredient list.

[1423] Step 5:

[1424] Collaboration with online supermarkets:

[1425] The terminal sends the ingredient list that the user has confirmed and edited to the server, and the server automatically places the order via the online supermarket's API. Once the order details are confirmed, the server converts the order details into a QR code and sends it to the terminal. The input for this step is the confirmed and edited ingredient list, and the output is the QR-coded order information.

[1426] Step 6:

[1427] Receiving and storing materials:

[1428] The user inputs the delivered ingredients into the terminal and sends it to the server. The server stores this purchased ingredient information in a database. The input of this step is the purchased ingredient information entered by the user, and the output is the information stored in the database.

[1429] Step 7:

[1430] Cooking instructions provided:

[1431] The server obtains specific cooking instructions and video links based on the purchased ingredient information via the cooking app's API and sends them to the user's device. The device then displays the cooking instructions to the user. The input for this step is the purchased ingredient information, and the output is the provided cooking instructions and video links.

[1432] Step 8:

[1433] Manage leftover ingredients and create your next recipe:

[1434] The server analyzes and records leftover ingredient information based on the purchase history and cooking history sent by the user. Based on this, it generates recipes to minimize waste when generating the next menu. The input for this step is the purchase history and cooking history, and the output is the recipe reflected in the next menu generation.

[1435] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1436] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1437] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1438] [Fourth embodiment]

[1439] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1440] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1441] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1442] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1443] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1444] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1445] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1446] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1447] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1448] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1449] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1450] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1451] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1452] As a form for implementing the present invention, we will specifically explain a series of systems that generate meal menus based on health information and ingredient inventory information, procure ingredients through collaboration with an online supermarket, and provide cooking methods.

[1453] User registration and initial settings

[1454] When a user uses the system for the first time, they first enter their profile information (age, gender, allergy information, calorie intake goal, etc.) into the device. The device sends the entered information to the server, which stores it. Next, the user enters information about food ingredients in their home (e.g., rice, chicken, vegetables, etc.) into the device, which also sends this information to the server and stores it.

[1455] Meal Menu Generation

[1456] The server retrieves the stored personal information and food inventory information and uses the AI ​​model to generate a meal menu for several days. For example, it generates a menu of "chicken and tomato soup, salad, and brown rice" based on a calorie intake target of 1500 kcal. The generated menu is sent from the server to the user's device.

[1457] Generate and order materials lists

[1458] Based on the received menu, the terminal displays a list of required ingredients (e.g., "300g chicken, 2 tomatoes, salad vegetables, 2 cups brown rice") to the user. The user can check this list and edit it as necessary (e.g., change the amount, add, or delete). The confirmed ingredient list is sent as an order from the terminal via the online supermarket's API. The online supermarket receives the order and sets the delivery date and time and collection method.

[1459] QR Code Generation and Distribution

[1460] Once the order details are confirmed, the server converts the information into a QR code and sends it to the user's device. The user can then use the QR code received on their device to confirm receipt at the online supermarket.

[1461] Save purchased ingredients information and provide cooking instructions

[1462] The user inputs the delivered ingredients into the device and sends that information to the server. The server saves the purchased ingredient information and, based on the saved data, retrieves specific cooking instructions and video links via the cooking app's API. This information is sent from the server to the user's device, and the user can cook while referring to the cooking instructions displayed on the device.

[1463] Managing leftover ingredients and creating your next recipe

[1464] The server analyzes and records leftover ingredient information based on the user's purchase and cooking history. This information is reflected in the next menu generation and creates recipes to minimize waste. For example, it might suggest a new salad recipe using unused tomatoes.

[1465] Specific examples

[1466] When a female user in her 30s uses the system for the first time, she first enters her profile (e.g., female in her 30s, no allergies, calorie intake goal 1500 kcal) and her household food inventory (e.g., rice, chicken, tomatoes) into the device. Once this information is saved on the server, the AI ​​generates a menu of "chicken and tomato soup, salad, brown rice" based on that information and sends it to the user.

[1467] Once the user confirms and edits the ingredient list and submits the order to the online supermarket, the order is converted into a QR code and sent to them. After that, the user re-enters the received ingredient information, and a link to a cooking video with detailed cooking instructions is displayed on the device. The system aims to simplify daily meal preparation and promote health maintenance and efficient ingredient management.

[1468] The present invention reduces the burden of meal planning and food ingredient management on the user, enabling them to maintain a healthy diet.

[1469] The processing flow will be explained below.

[1470] Step 1:

[1471] The user enters profile information (age, gender, allergy information, calorie intake goal, etc.) into the terminal.

[1472] Step 2:

[1473] The terminal transmits the entered profile information to the server.

[1474] Step 3:

[1475] The server stores the received profile information in a database.

[1476] Step 4:

[1477] The user inputs information about food ingredients in the home into the terminal.

[1478] Step 5:

[1479] The terminal transmits the input food stock information to the server.

[1480] Step 6:

[1481] The server stores the received food stock information in a database.

[1482] Step 7:

[1483] The server uses an AI model to generate a meal menu for several days based on the saved profile information and food inventory information.

[1484] Step 8:

[1485] The server transmits the generated meal menu to the user terminal.

[1486] Step 9:

[1487] The terminal displays the meal menu received from the server to the user.

[1488] Step 10:

[1489] The user reviews the meal menu and edits the ingredient list as needed.

[1490] Step 11:

[1491] The terminal sends the confirmed list of ingredients to the online supermarket's API.

[1492] Step 12:

[1493] The online supermarket processes the order based on the received ingredient list and sets the delivery date and time and collection method.

[1494] Step 13:

[1495] The server receives the order details from the online supermarket and converts them into a QR code.

[1496] Step 14:

[1497] The server sends the generated QR code to the user's terminal.

[1498] Step 15:

[1499] The user receives the QR code on their device and uses it to receive the ingredients.

[1500] Step 16:

[1501] The user inputs the delivered food information into the terminal.

[1502] Step 17:

[1503] The terminal transmits information about the delivered ingredients to the server.

[1504] Step 18:

[1505] The server stores the received purchased ingredient information in a database.

[1506] Step 19:

[1507] The server connects to the cooking app's API based on the stored ingredient information to obtain specific cooking instructions and video links.

[1508] Step 20:

[1509] The server sends the acquired cooking instructions and video links to the user's terminal.

[1510] Step 21:

[1511] The user cooks the food while referring to the cooking instructions displayed on the terminal.

[1512] Step 22:

[1513] The server analyzes and records information about leftover ingredients based on the purchase history and cooking history sent by the user.

[1514] Step 23:

[1515] The server will take the remaining ingredient information into consideration when generating a new menu next time.

[1516] This series of processing steps streamlines meal preparation and health management for users.

[1517] Example 1

[1518] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1519] Conventional meal planning systems require users to plan their own meal menus, source the ingredients, and search for cooking methods, which is time-consuming. They also tend to waste ingredients and make efficient food management difficult. There is a need for a system that can solve these problems and allow users to easily maintain healthy meals.

[1520] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1521] In this invention, the server includes means for inputting and saving user information including health information, means for inputting and saving household food inventory information, means for generating a meal menu using a generative AI model based on the saved user information and food inventory information, means for displaying a list of necessary ingredients based on the generated meal menu, means for ordering ingredients via an interface of an online sales service, means for coding and distributing the order details, means for saving information on purchased ingredients and providing cooking instructions, and means for recording information on leftover ingredients and reflecting it in generating the next menu. This allows users to easily plan healthy meals and efficiently procure and manage the necessary ingredients.

[1522] "Health information" refers to health-related data such as a user's age, gender, allergy information, and calorie intake goal.

[1523] "User Information" means general data about individuals using the system, including health information.

[1524] "Food inventory information" is data on the types and quantities of food currently held in the household.

[1525] "Storage means" refers to the methods and techniques used to record and retain input information in a database.

[1526] A "generative AI model" is an algorithm or system that uses artificial intelligence technology to generate specific information from input data.

[1527] A "meal menu" is a list of specific meals that a user plans to consume.

[1528] An "ingredient list" is a list of ingredients and quantities needed to prepare a particular meal.

[1529] An "interface for online retail services" is a means of connection to an online retailer's system for ordering products.

[1530] "Encoding" refers to converting data into a format such as a barcode or QR code.

[1531] "Purchased ingredient information" is data regarding the types and quantities of ingredients actually purchased by the user.

[1532] "Means for providing cooking instructions" refers to methods or services that allow users to refer to specific cooking procedures.

[1533] "Leftover ingredient information" is data regarding the type and amount of ingredients that were not used after cooking.

[1534] As an embodiment of the present invention, we will specifically explain a system that generates meal menus based on a user's health information and home food inventory information, procures the necessary ingredients through an online sales service, and provides cooking instructions. This system is composed of a server, terminals, and users.

[1535] First, when a user uses the system for the first time, they use their device to enter their own health information (e.g., age, gender, allergy information, calorie intake goal). The device sends this information to the server using an HTTP request, and the server stores the received information in a database. Next, the user enters information about food ingredients in their home (e.g., rice, chicken, vegetables, etc.), and the device again sends this to the server for storage. The database used can be, for example, MySQL.

[1536] Next, the server uses a generative AI model (e.g., OpenAI's GPT-4) to generate a meal menu for several days based on the stored health information and food inventory information. For example, if the calorie intake goal is 1500 kcal, a menu such as "chicken and tomato soup, salad, and brown rice" can be generated. The generated menu is sent from the server to the user's device.

[1537] The device generates a list of required ingredients (e.g., "300g chicken, 2 tomatoes, salad vegetables, 2 cups brown rice") based on the received menu and displays it to the user. When the user checks and edits the list of ingredients and taps the "Order" button, the device sends the order data using the API of the online sales service. For example, a general REST API can be used here. The online sales service receives the order and sets the delivery date and time and collection method.

[1538] Once the order details are confirmed, the server generates a QR code based on that information and sends it to the user's device. The user then uses this QR code to confirm receipt of the item through the online sales service. QR codes can be generated using the zxing library, for example.

[1539] The user enters the received ingredient information into the device, which then sends the information back to the server. The server stores the received data and retrieves specific cooking instructions and video links via the cooking assistant app's API (e.g., a cooking recipe service). The retrieved information is then sent from the server to the user's device and displayed.

[1540] Furthermore, the server analyzes and records leftover ingredient information based on the user's purchase and cooking history. By incorporating this information into the next menu generation, it is possible to create recipes that minimize food waste. For example, it can suggest new salad recipes using unused tomatoes.

[1541] Examples and prompts

[1542] As a concrete example, let's consider the case of a female user in her 30s using this system for the first time. First, the user enters her profile (e.g., female in her 30s, no allergies, calorie intake goal 1500 kcal) and household ingredient inventory (e.g., rice, chicken, tomatoes) into her device. This information is saved on the server, and the generative AI model uses it to generate a menu of "chicken and tomato soup, salad, and brown rice" and sends it to the user's device. The user checks and edits the ingredient list and submits the order to the online sales service, where the order details are sent as a QR code. The user then re-enters the received ingredient information, and a link to a cooking video providing detailed cooking instructions is displayed on the user's device.

[1543] An example prompt is:

[1544] "Generate a healthy meal plan for a woman in her 30s with no allergies and a calorie intake goal of 1500 kcal. The ingredients in the household are rice, chicken, and tomatoes."

[1545] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1546] Step 1: User registration and initial setup

[1547] The user launches the application and enters health information such as age, gender, allergy information, and calorie intake goal into the device. The entered information is sent by the device to the server via an HTTP request. The server stores the received health information in a database (e.g., MySQL). Next, the user enters the food inventory in the home (e.g., rice, chicken, vegetables, etc.), and the device similarly sends this to the server and stores it in the database.

[1548] Input: age, gender, allergy information, calorie intake goal, food inventory information.

[1549] Output: Health information and food inventory information stored in a database.

[1550] Specific operation: The user enters information into the input field of the smartphone app and taps the "Save" button. This operation causes the device to send the information to the server. The server receives the data and stores it in a database.

[1551] Step 2: Meal menu generation

[1552] The server retrieves the stored health information and food inventory information from the database and sends a prompt to a generative AI model (e.g., OpenAI's GPT-4). The generative AI model generates a meal menu (e.g., "chicken and tomato soup, salad, and brown rice") and sends it to the server. The server stores the received menu in a database and sends it to the user's device.

[1553] Input: Health information, food inventory information, prompt text.

[1554] Output: The generated meal menu.

[1555] Specific operation: The server creates a prompt and sends it to the AI ​​model. The AI ​​model generates a menu and returns it to the server. The server saves the menu and sends it to the device. The user's device displays the menu via a push notification.

[1556] Step 3: Generate materials list and order

[1557] Based on the menu received, the terminal generates a list of required ingredients and displays it to the user. The user checks the list of ingredients, makes any necessary edits (e.g., changes to quantities, adding or deleting items), and confirms the "order." The terminal then sends the order data via the API of the online sales service. The online sales service receives the order data and sets the delivery date and time, etc.

[1558] Input: The generated meal menu.

[1559] Output: Generated material list, order details.

[1560] Specific operation: The terminal analyzes the menu and generates an ingredient list. The user checks and edits the list. After confirming the order, the terminal sends the order to the online sales service's API. The online sales service receives and processes the order.

[1561] Step 4: Generate and distribute QR codes

[1562] Once the order details are confirmed, the server generates a QR code based on the order information and sends it to the user's device. The user then uses this QR code when picking up the item from the online shopping service.

[1563] Input: Order details.

[1564] Output: QR code.

[1565] Specific operation: The server analyzes the order details and generates a QR code (e.g., using the zxing library). The generated QR code URL is sent to the user's device. The user displays the QR code on their device and uses it to receive the item.

[1566] Step 5: Save purchased ingredients and provide cooking instructions

[1567] The user enters the received ingredient information into the device and sends it to the server. The server stores the received information in a database and obtains specific cooking instructions and video links via the cooking assistant app's API (e.g., a cooking recipe service). The server sends this information to the user's device, which then displays it to the user.

[1568] Input: Received ingredient information.

[1569] Output: Specific cooking instructions and video links.

[1570] Specific operation: The user enters the ingredients they have purchased into their device and taps the "Save" button. The device sends the information to the server. The server saves it in a database, then calls the cooking app's API to retrieve the cooking information. The retrieved cooking instructions are sent to the user's device, which displays them to the user.

[1571] Step 6: Manage leftover ingredients and create your next recipe

[1572] The server analyzes and records leftover ingredient information based on the purchase and cooking history sent by the user. This information is reflected in the next menu generation and suggests recipes that minimize waste.

[1573] Input: purchase history, cooking history.

[1574] Output: Information on leftover ingredients, next recipe.

[1575] Specific operation: The server retrieves historical information from the database and uses the AI ​​model to calculate the remaining amount of unused ingredients. The calculation result is included in the next prompt and sent to the AI ​​model. The model generates a new menu, which the server saves in the database. The generated menu is then notified to the user's device.

[1576] (Application example 1)

[1577] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1578] In modern society, personal health management and efficient dietary habits are important issues. In particular, the lack of a system that centrally manages multiple elements, such as generating efficient meal menus based on health information and food inventory information, online ordering, and providing cooking instructions, is a problem. There is also a need for a method to provide menus optimized for each individual's health status and use ingredients without waste based on such menus.

[1579] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1580] In this invention, the server includes means for inputting and saving personal information including health information, means for inputting and saving information on household food inventory, means for generating a meal menu using the saved personal information and food ingredient information, means for displaying a list of necessary ingredients based on the generated meal menu, means for ordering ingredients via an online supermarket API, means for converting the order content into a QR code and distributing it, means for saving information on purchased ingredients and providing cooking instructions, means for recording information on leftover ingredients and reflecting it in the next menu generation, means for providing instructions to a smartphone app based on the generated meal menu and food ingredient information, means for generating prompts, and means for generating an optimal meal menu using a generative AI model. This enables the provision of individually optimized healthy menus, the use of ingredients without waste, and centralized management of online orders.

[1581] "Health information" refers to information related to an individual's health, such as an individual's age, gender, allergy information, and calorie intake goal.

[1582] "Personal information" is information necessary to identify a specific individual, including health information.

[1583] "Food inventory information" refers to information about the types and amounts of food ingredients currently stored in the home.

[1584] "Meal Menu" refers to a combination of dishes or a menu suggested based on personal health information and food inventory information.

[1585] "Ingredients list" refers to a list of the types and quantities of ingredients needed to prepare a particular meal.

[1586] An "online supermarket" is an online retail store where you can order and purchase food and daily necessities via the Internet.

[1587] "API" is an abbreviation for Application Program Interface, and refers to an interface for linking functions between software.

[1588] "QR code" refers to a two-dimensional barcode that allows for easy electronic acquisition of information.

[1589] "Cooking instructions" refers to information such as steps or tips for making a particular dish, or a video link showing how to do so.

[1590] A "prompt" is a message or instruction that requests some input or response from the user.

[1591] A "generative AI model" refers to an artificial intelligence model that generates optimal output based on input data.

[1592] "Smartphone app" refers to a software application that runs on a smartphone.

[1593] As a form for implementing this invention, we will specifically explain a series of systems that generate meal menus based on health information and ingredient inventory information, procure ingredients through collaboration with an online supermarket, and provide cooking methods.

[1594] User registration and initial settings

[1595] When a user uses the system for the first time, they first enter their profile information (age, gender, allergy information, calorie intake goal, etc.) into the device. The device sends the entered information to the server, which stores it. Next, the user enters information about food ingredients in their home (e.g., rice, chicken, vegetables, etc.) into the device, which also sends this information to the server and stores it.

[1596] Meal Menu Generation

[1597] The server retrieves the stored personal information and food inventory information and uses the generative AI model to generate several days' worth of meal menus. For example, it generates a menu of "chicken and tomato soup, salad, and brown rice" based on a calorie intake target of 1500 kcal. The generated menu is sent from the server to the user's device.

[1598] Generate and order materials lists

[1599] Based on the received menu, the terminal displays a list of required ingredients (e.g., "300g chicken, 2 tomatoes, salad vegetables, 2 cups brown rice") to the user. The user can check this list and edit it as necessary (for example, change the amount, add, or delete). The confirmed ingredient list is sent as an order from the terminal via the online supermarket's API. The online supermarket receives the order and sets the delivery date, time, and collection method.

[1600] QR Code Generation and Distribution

[1601] Once the order details are confirmed, the server converts the information into a QR code and sends it to the user's device. The user can then use the QR code received on their device to confirm receipt at the online supermarket.

[1602] Save purchased ingredients information and provide cooking instructions

[1603] The user inputs the delivered ingredients into the device and sends that information to the server. The server saves the purchased ingredient information and, based on the saved data, retrieves specific cooking instructions and video links via the cooking app's API. This information is sent from the server to the user's device, and the user can cook while referring to the cooking instructions displayed on the device.

[1604] Managing leftover ingredients and creating your next recipe

[1605] The server analyzes and records leftover ingredients based on the user's purchase and cooking history. This information is reflected in the next menu generation and creates recipes to minimize waste. For example, it might suggest a new salad recipe using unused tomatoes.

[1606] Specific examples

[1607] When a female user in her 30s uses the system for the first time, she first enters her profile (e.g., female in her 30s, no allergies, calorie intake goal 1500 kcal) and her household food inventory (e.g., rice, chicken, tomatoes) into the device. Once this information is saved on the server, the generative AI model uses it to create a menu of "chicken and tomato soup, salad, brown rice" and sends it to the user.

[1608] Once the user checks and edits the ingredient list and submits the order to the online supermarket, the order details are sent as a QR code. After that, the user re-enters the received ingredient information, and a link to a cooking video providing detailed cooking instructions is displayed on the device.

[1609] Prompt Sentence Examples

[1610] User Information:

[1611] Age: 30

[1612] Gender: Female

[1613] Calorie intake goal: 1500 kcal

[1614] Allergy Information: None

[1615] Ingredients availability information:

[1616] Rice: 2 cups

[1617] Chicken: 1 piece

[1618] Tomatoes: 3

[1619] Please suggest a healthy meal plan based on the above information.

[1620] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1621] Step 1:

[1622] The user enters their profile information into the device, which then sends the information (such as age, gender, allergy information, and calorie intake goal) to the server, where it is stored in a database.

[1623] Input: Age, gender, allergy information, calorie intake goal

[1624] Output: Personal information stored in the database

[1625] Specific operation: The user enters information using a smartphone app and presses the "Save" button. The device sends the entered information as a request to the server's API, and the server saves the information in a database.

[1626] Step 2:

[1627] The user inputs information about the food inventory in their home into the terminal. The input data (e.g., rice, chicken, tomatoes, etc.) is sent from the terminal to the server, which then stores it in a database.

[1628] Input: Ingredient information such as rice, chicken, and tomatoes

[1629] Output: Ingredient inventory information stored in the database

[1630] Specific operation: The user enters the inventory and presses the "Save" button. The terminal sends the information as a request to the server's API, and the server saves the information in the database.

[1631] Step 3:

[1632] The server retrieves the stored personal information and food inventory information and generates several days' worth of meal menus using a generative AI model.

[1633] Input: Personal information, food inventory information

[1634] Output: Generated meal menus for several days

[1635] Specific operation: The server inputs personal information and food inventory information into the AI ​​model and generates the following prompt: "User information: Age: 30, Gender: Female, Calorie intake goal: 1500 kcal, Allergy information: None, Food inventory information: Rice: 2 cups, Chicken: 1, Tomatoes: 3. Based on the above information, please suggest a healthy meal menu." This is input into the generative AI model to obtain a meal menu.

[1636] Step 4:

[1637] Based on the received menu, the device displays a list of ingredients (e.g., "300g chicken, 2 tomatoes, salad greens, 2 cups brown rice") to the user, who can then review and edit the list if necessary.

[1638] Input: Generated meal menu

[1639] Output: Materials list

[1640] Specific operation: The terminal retrieves and displays the menu, and the user confirms and edits it. The terminal visualizes and displays the ingredient list.

[1641] Step 5:

[1642] The confirmed ingredients list is then sent from the device to the online supermarket via its API, which then receives the order and sets the delivery date and time and pick-up method.

[1643] Input: Edited materials list

[1644] Output: Order information and delivery date / time / pickup method settings

[1645] Specific operation: The terminal sends an order request to the online supermarket's API, the server processes the order, and the terminal displays an order confirmation.

[1646] Step 6:

[1647] Once the order details are confirmed, the server converts the information into a QR code and sends it to the user's device. The user then uses the QR code received on their device to confirm receipt at the online supermarket.

[1648] Input: Order Information

[1649] Output: QR code

[1650] Specific operation: The order information is converted using a QR code generation library, and the generated QR code is sent to the device. The user scans the QR code when receiving the order.

[1651] Step 7:

[1652] The user inputs the delivered ingredients into the device and sends the information to the server. The server saves the purchased ingredients and retrieves specific cooking instructions and video links based on the saved data via the cooking app's API.

[1653] Input: Delivered food information

[1654] Output: Cooking instructions and video links

[1655] Specific operation: The user inputs the delivered ingredients, and the device sends the information to the server. The server accesses the cooking app's API to retrieve the information and displays it on the user's device.

[1656] Step 8:

[1657] The server analyzes and records leftover ingredients based on the purchase and cooking history sent by the user. This information is reflected in the next menu generation, and recipes are created to minimize waste.

[1658] Input: purchase history, cooking history

[1659] Output: Recipe reflected in next menu generation

[1660] Specific operation: The server analyzes purchase and cooking history data, records leftover ingredients, and reflects them in the next menu generation. If necessary, it generates new prompts and inputs them into the generative AI model to obtain recipes.

[1661] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1662] As a form for implementing the present invention, we will specifically explain a system that generates meal menus based on health information and ingredient inventory information, procures ingredients through collaboration with an online supermarket, and provides cooking methods, all of which are combined with an emotion engine that recognizes the user's emotions.

[1663] User registration and initial settings

[1664] When a user uses the system for the first time, they first enter their profile information (age, gender, allergy information, calorie intake goal, etc.) into the device. The device then sends the entered information to the server, which stores it. The user then enters information about the food ingredients in their home (e.g., rice, chicken, vegetables, etc.) into the device, which then sends this information to the server and stores it. In addition, an interface that allows the emotion engine to collect the user's emotion information is also included as part of the initial setup.

[1665] Manipulating the Emotion Engine

[1666] The emotion engine continuously collects data such as the user's facial expressions, voice, and text messages to recognize their emotions, and then analyzes and transmits emotional information such as their current mood and stress level to the server.

[1667] Meal Menu Generation

[1668] The server uses an AI model to generate several days' worth of meal menus based on stored personal information, ingredient inventory information, and emotional information provided by the emotion engine. For example, if the server detects that the user is feeling stressed, it generates a menu using ingredients that have a relaxing effect. The generated menu is then sent from the server to the user's device.

[1669] Generate and order materials lists

[1670] Based on the received menu, the terminal displays a list of required ingredients (e.g., "300g chicken, 2 tomatoes, salad vegetables, 2 cups brown rice") to the user. The user can check this list and edit it as necessary (e.g., change the amount, add, or delete). The confirmed ingredient list is sent as an order from the terminal via the online supermarket's API. The online supermarket receives the order and sets the delivery date and time and collection method.

[1671] QR Code Generation and Distribution

[1672] Once the order details are confirmed, the server converts the information into a QR code and sends it to the user's device. The user can then use the QR code received on their device to confirm receipt at the online supermarket.

[1673] Save purchased ingredients information and provide cooking instructions

[1674] The user inputs the delivered ingredients into the device and sends that information to the server. The server saves the purchased ingredient information and, based on the saved data, retrieves specific cooking instructions and video links via the cooking app's API. This information is sent from the server to the user's device, and the user can cook while referring to the cooking instructions displayed on the device.

[1675] Managing leftover ingredients and creating your next recipe

[1676] The server analyzes and records leftover ingredient information based on the user's purchase and cooking history. Furthermore, this information, along with data from the emotion engine, is taken into account when generating the next menu, creating recipes to minimize waste. For example, it can suggest new salad recipes using unused tomatoes.

[1677] Specific examples

[1678] A female user in her 30s uses the system for the first time and enters her profile information (e.g., female in her 30s, no allergies, calorie intake goal 1500 kcal) and her household food inventory (e.g., rice, chicken, tomatoes) into the device. The emotion engine detects that the user is feeling stressed. This information is also sent to the server, and the AI ​​uses it to generate a menu of "relaxing chicken and tomato soup, salad, and brown rice" and sends it to the user.

[1679] Once the user confirms and edits the ingredient list and submits the order to the online supermarket, the order details are sent as a QR code. When the user re-enters the received ingredient information, a link to a cooking video with specific cooking instructions is displayed on the device. The system aims to simplify daily meal preparation and promote health maintenance and efficient ingredient management.

[1680] The present invention reduces the burden of meal planning and food ingredient management on the user, and further makes it possible to propose menus that match the user's mood based on emotional information, thereby enabling the user to maintain a healthy diet.

[1681] The processing flow will be explained below.

[1682] Step 1:

[1683] The user enters profile information (age, gender, allergy information, calorie intake goal, etc.) into the terminal.

[1684] Step 2:

[1685] The terminal transmits the entered profile information to the server.

[1686] Step 3:

[1687] The server stores the received profile information in a database.

[1688] Step 4:

[1689] The user inputs information about food ingredients in the home into the terminal.

[1690] Step 5:

[1691] The terminal transmits the input food stock information to the server.

[1692] Step 6:

[1693] The server stores the received food stock information in a database.

[1694] Step 7:

[1695] The emotion engine collects the user's emotional information (facial expressions, voice, text messages, etc.) and recognizes emotions.

[1696] Step 8:

[1697] The emotion engine sends the recognized emotion information to the server.

[1698] Step 9:

[1699] The server acquires the stored personal information, food stock information, and emotion information.

[1700] Step 10:

[1701] The server uses an AI model to generate a meal menu for several days that takes into account emotional information. For example, if the user is feeling stressed, it will generate a menu with a relaxing effect.

[1702] Step 11:

[1703] The server transmits the generated meal menu to the user terminal.

[1704] Step 12:

[1705] The terminal displays the meal menu received from the server to the user.

[1706] Step 13:

[1707] The user reviews the meal menu and edits the ingredient list as needed.

[1708] Step 14:

[1709] The terminal sends the confirmed list of ingredients to the online supermarket's API.

[1710] Step 15:

[1711] The online supermarket processes the order based on the received ingredient list and sets the delivery date and time and collection method.

[1712] Step 16:

[1713] The server receives the order details from the online supermarket and converts them into a QR code.

[1714] Step 17:

[1715] The server sends the generated QR code to the user's terminal.

[1716] Step 18:

[1717] The user receives the QR code on their device and uses it to receive the ingredients.

[1718] Step 19:

[1719] The user inputs the delivered food information into the terminal.

[1720] Step 20:

[1721] The terminal transmits information about the delivered ingredients to the server.

[1722] Step 21:

[1723] The server stores the received purchased ingredient information in a database.

[1724] Step 22:

[1725] Based on the saved data, the server obtains specific cooking instructions and video links via the cooking app's API.

[1726] Step 23:

[1727] The server sends the acquired cooking instructions and video links to the user's terminal.

[1728] Step 24:

[1729] The user cooks the food while referring to the cooking instructions displayed on the terminal.

[1730] Step 25:

[1731] The server analyzes the purchase history and cooking history sent by the user and records information about leftover ingredients.

[1732] Step 26:

[1733] The server takes into account the remaining ingredients and emotion information and reflects them in the next menu generation. For example, it generates a menu using unused tomatoes.

[1734] This series of processing steps streamlines the user's meal preparation and health management, while providing a balanced meal plan based on emotional information.

[1735] Example 2

[1736] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1737] Conventional meal planning systems only considered personal health information and household food inventory information, and were unable to suggest menus that took the user's emotional state into account. This made it difficult to suggest appropriate menus when the user was feeling stressed or in a particular mood, limiting their ability to provide comprehensive health management for the user. Furthermore, their management of food waste to a minimum was insufficient, leaving a need for more efficient food use.

[1738] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1739] In this invention, the server includes means for inputting and saving personal information including health information, means for inputting and saving household ingredient inventory information, means for generating a meal menu using the saved personal information and ingredient inventory information, means for displaying a list of ingredients required based on the generated meal menu, means for ordering ingredients via an API of an online sales service, means for converting the order details into a QR code and distributing it, means for saving information on purchased ingredients and providing cooking instructions, means for recording information on leftover ingredients and reflecting it in generating the next menu, and means for collecting, analyzing information on the user's emotions and reflecting it in generating the meal menu. This makes it possible to propose an appropriate meal menu based on the user's health condition and emotional state, and to reduce waste by using ingredients efficiently.

[1740] "Health information" is data about the user's physical and health status, such as age, gender, allergy information, and calorie intake goals.

[1741] "Personal Information" is any data about a user, including profile information and health information.

[1742] "Home food ingredient inventory information" is information about the types and quantities of ingredients currently stored in the user's home.

[1743] A "meal menu" is a list of meals that the user plans to consume, generated based on the user's personal information and home food inventory information.

[1744] The "ingredient list" is a list of ingredients required based on the generated meal menu.

[1745] An "API for online sales services" is a program interface for ordering products online.

[1746] A "QR code" is a two-dimensional code that visually conveys information about an order.

[1747] "Purchased ingredient information" is information about ingredients purchased by a user via an online sales service.

[1748] "Cooking instructions" is information about the steps and techniques for cooking using purchased ingredients.

[1749] "Leftover ingredient information" is information about ingredients that were left over from the previous cooking session and were not used.

[1750] "Emotion information" is data relating to the user's emotional state obtained from facial expressions, voice, text, etc.

[1751] The present invention combines a series of systems that generate meal menus based on a user's health information and household food inventory information, procure ingredients in cooperation with online sales services, and provide cooking methods with an emotion engine that recognizes the user's emotion information. Specific embodiments of this system are described below.

[1752] First, when using the device for the first time, the user enters their profile information (age, gender, allergy information, calorie intake goal, etc.) into the device, which then sends and stores this information on the server. Next, the user enters information about the food ingredients in their home (rice, chicken, tomatoes, etc.) into the device, which then sends this information to the server and stores it. The initial settings for the emotion engine are also performed on the device, and an interface is built to collect the user's emotion information.

[1753] The emotion engine collects emotional data from users through facial expression analysis, voice recognition, and text analysis, and analyzes it in real time. The analysis results are sent to a server to identify their state of stress and other factors.

[1754] The server combines the stored personal information, food inventory information, and emotional information provided by the emotion engine to generate a meal menu for several days using an AI model. For example, if the server determines that the user is feeling stressed, it will suggest a menu using ingredients that have a relaxing effect. This generated menu is then sent from the server to the user's device.

[1755] The user's device creates a list of ingredients based on the received menu and displays it to the user. Once the user confirms and edits the ingredients list, the confirmed list is sent to the online supermarket via its API. The online supermarket receives the order and sets the delivery date and time and collection method.

[1756] Once the order details are confirmed, the server converts the information into a QR code and sends it to the user's device. The user uses the QR code on their device to confirm receipt at the online supermarket. Information about the delivered ingredients is again entered into the device and sent to the server. Based on this information, the server obtains specific cooking instructions and a link to a cooking video via the cooking assistance application's API and sends it to the user's device. The user cooks while referring to this information.

[1757] The server also analyzes the user's purchase and cooking history and records any leftover ingredients. The next time the server generates a menu, it takes this information and emotional information into account to create recipes that minimize waste. For example, it might suggest a new salad recipe using unused tomatoes.

[1758] The present invention makes it possible to propose an appropriate meal menu according to the user's health condition and emotional state, and also reduces waste by using ingredients more efficiently.

[1759] Specific examples

[1760] A female user in her 30s uses the system for the first time and enters her profile information (e.g., female in her 30s, no allergies, calorie intake goal 1500 kcal) and her household food inventory (e.g., rice, chicken, tomatoes) into the device. The emotion engine detects that the user is feeling stressed. This information is also sent to the server, and the AI ​​uses it to generate a menu of "relaxing chicken and tomato soup, salad, and brown rice" and sends it to the user.

[1761] Prompt Sentence Examples

[1762] "A woman in her 30s with no allergies and a calorie intake goal of 1500 kcal. She has rice, chicken, and tomatoes at home. Please suggest a meal menu that will help the user relax when they are feeling stressed."

[1763] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1764] Step 1: User registration and initial setup

[1765] When a user uses the device for the first time, they enter their profile information (e.g., age, gender, allergy information, calorie intake goal, etc.). The entered information is sent from the device to the server, which stores it in a database. Next, the user enters information about the food inventory in their home (e.g., 1 kg of rice, 500 g of chicken, 3 tomatoes) into the device, which also sends this information to the server and stores it. The device then initializes the emotion engine and builds an interface to collect the user's emotion information.

[1766] input:

[1767] Profile information (age, gender, allergy information, calorie intake goal, etc.)

[1768] Home food inventory information (rice, chicken, tomatoes, etc.)

[1769] output:

[1770] Personal information and food inventory information stored on the server

[1771] The initial settings of the emotion engine are complete

[1772] Specific behavior:

[1773] The terminal receives input information from the user and sends it to the server via the API.

[1774] The server stores the received data in a database.

[1775] The terminal displays an emotion engine setting wizard and asks the user for permission to collect emotion data.

[1776] Step 2: Manipulating the Emotion Engine

[1777] The emotion engine collects emotion data through facial expression analysis, voice recognition, and text analysis. This data is analyzed in real time to extract the user's current emotion information (e.g., stress level) and send it to the server.

[1778] input:

[1779] User facial expressions, voice, and text data

[1780] output:

[1781] Analyzed emotional information (e.g., whether the user is feeling stressed)

[1782] Specific behavior:

[1783] The emotion engine captures the user's facial expressions and voice via the camera and microphone and analyzes text messages.

[1784] The emotion engine uses machine learning algorithms to analyze the emotion data and send the results to a server.

[1785] The server integrates the received emotion information into a user profile and stores it.

[1786] Step 3: Meal menu generation

[1787] The server uses an AI model to generate a meal menu based on stored personal information, food inventory information, and emotional information. For example, if the server detects that the user is feeling stressed, it will generate a menu using ingredients that have a relaxing effect. The generated menu is then sent from the server to the user's device.

[1788] input:

[1789] personal information

[1790] Food inventory information

[1791] emotional information

[1792] output:

[1793] Generated meal menu (e.g., relaxing chicken and tomato soup, salad, and brown rice)

[1794] Specific behavior:

[1795] The server inputs the stored data into an AI model, which then generates a meal menu.

[1796] The server sends the generated menu to the terminal.

[1797] Step 4: Generate materials list and order

[1798] The device creates a list of required ingredients based on the received menu and displays it to the user. The user can check this list and change the quantities or add or delete items. The confirmed list is then sent from the device via the API of the online sales service. The online sales service receives the order details and sets the delivery date and time and collection method.

[1799] input:

[1800] Generated meal menu

[1801] output:

[1802] Required Materials List

[1803] Submitted order information

[1804] Specific behavior:

[1805] The device analyzes the meal menu and automatically generates a list of necessary ingredients.

[1806] The user edits this list, and after reviewing it, sends it to the online sales service via API.

[1807] The online sales service receives the order information and begins processing it.

[1808] Step 5: Generate and distribute QR codes

[1809] The server converts the order details into a QR code and sends it to the user's device, which the user can use to confirm receipt of the order through the online sales service.

[1810] input:

[1811] Order details

[1812] output:

[1813] QR code

[1814] Specific behavior:

[1815] The server encodes the order details into a QR code and sends the generated QR code to the terminal.

[1816] The user displays the QR code on their device and scans it upon receipt.

[1817] Step 6: Save purchased ingredients and provide cooking instructions

[1818] The user inputs the delivered ingredients into the device and sends that information to the server. The server saves the purchased ingredient information and, based on the saved data, retrieves specific cooking instructions and video links via the cooking assistance application's API. This information is sent from the server to the user's device, and the user cooks while referring to the cooking instructions displayed on the device.

[1819] input:

[1820] Purchased food information

[1821] output:

[1822] Saved ingredient information

[1823] Cooking methods and cooking video links

[1824] Specific behavior:

[1825] The user enters the received ingredient information into the terminal and sends it to the server via the API.

[1826] The server uses the stored ingredient information to connect to the API of the cooking assistance application and obtain cooking instructions and video links.

[1827] The acquired information is sent to the terminal so that the user can view and implement it.

[1828] Step 7: Manage leftover ingredients and create your next recipe

[1829] The server analyzes and records leftover ingredient information based on the purchase and cooking history sent by the user. Taking this information and data from the emotion engine into consideration, the server generates recipes that minimize waste and are reflected in the next menu creation.

[1830] input:

[1831] Purchase history

[1832] Cooking history

[1833] emotional information

[1834] output:

[1835] Leftover food information

[1836] Menu for next time

[1837] Specific behavior:

[1838] The server analyzes the data sent by the user and identifies any leftover ingredients.

[1839] The remaining ingredients will be taken into consideration when creating the next recipe, and a menu will be created to minimize waste.

[1840] The new menu is sent to the user terminal.

[1841] (Application example 2)

[1842] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1843] In modern life, it is extremely difficult to efficiently manage daily meal preparation while maintaining a healthy diet. In particular, there is a demand for home food ingredient management and flexible menu suggestions that respond to the user's mood. Furthermore, while meeting these demands, it is also necessary to achieve smooth food procurement through integration with online supermarkets. To solve these issues, it is essential to develop a system that can centrally manage the entire process and is easy for users to use.

[1844] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1845] In this invention, the server includes means for inputting and saving personal information including health information, means for inputting and saving information about home food ingredient inventory, means for generating a meal menu using the saved personal information and ingredient information, means for displaying a list of ingredients needed based on the generated meal menu, means for ordering ingredients via an online supermarket API, means for converting the order details into a QR code and distributing it, means for saving information about purchased ingredients and providing cooking instructions, means for recording information about remaining ingredients and reflecting this in generating the next menu, means for recognizing and collecting user emotional information, means for analyzing the emotional information and optimizing the meal menu based on the analysis results, means for acquiring the emotional information on a smartphone, and means for displaying the suggested meal menu via an application used in the virtual store. This allows for the suggestion of an optimal meal menu based on the user's health condition and emotions, allowing for smooth procurement of ingredients and efficient cooking based on the menu.

[1846] "Health information" refers to information related to the user's health condition and nutritional balance, such as age, gender, allergy information, and calorie intake goal.

[1847] "Personal Information" refers to information that is individually associated with a user, specifically the user's profile information, health information, and emotional information.

[1848] "Inventory information for ingredients" refers to information indicating the types and quantities of ingredients currently available in the home, such as rice, chicken, and vegetables.

[1849] "Emotional information" refers to information that indicates the user's current emotional state, such as stress level and mood.

[1850] "Virtual store" refers to a platform that provides a virtual store experience via the Internet, where users can purchase products.

[1851] "Emotion engine" refers to the algorithms and mechanisms for recognizing and collecting a user's emotional state by analyzing data such as facial expressions, voice, and text messages.

[1852] A "generative AI model" refers to an artificial intelligence algorithm that provides a rational output based on input data. Examples include GPT-4 and BERT.

[1853] A "prompt sentence" refers to the input sentence that is fed into a generative AI model, which then generates an output based on that sentence.

[1854] "Diet Menu" refers to a list of meals that are recommended for the user to consume, generated based on health and emotional information.

[1855] "Ingredient list" refers to a list of ingredients needed to cook a meal based on the menu. For example, "300g chicken, 2 tomatoes."

[1856] "Net supermarket" refers to a retail service that allows users to order food and daily necessities online. Users can purchase the products they need via the Internet.

[1857] An "API" is an interface that allows different software systems to share functionality. Specifically, you submit an order through the online supermarket's API.

[1858] A "QR code" is a two-dimensional barcode that visually encodes digital information and can be scanned with a smartphone or other device.

[1859] "Cooking method" refers to the steps for making a dish using specified ingredients. Specifically, it can be a recipe or a link to a cooking video.

[1860] A "cooking app" refers to an application that provides cooking methods and recipes that users can refer to to prepare a dish.

[1861] User registration and initial settings

[1862] When a user uses the system for the first time, they enter their profile information (age, gender, allergy information, calorie intake goal, etc.) into the device using their smartphone, and this information is sent to the server. Next, they enter information about their household food inventory (e.g., rice, chicken, vegetables, etc.) into the device, and this information is also sent and saved to the server. An interface that allows the emotion engine to collect the user's emotion information is also included as part of the initial setup.

[1863] Manipulating the Emotion Engine

[1864] The emotion engine uses the smartphone's camera and microphone to continuously collect the user's facial expressions and voice to recognize their emotions. This information is then analyzed to determine their current mood and stress level, and the results are sent to the server. The user's emotional data includes stress, relaxation, happiness, etc.

[1865] Generate a meal menu

[1866] The server generates several days' worth of meal menus using a generative AI model (e.g., GPT-4, BERT) based on stored personal information, ingredient inventory information, and emotional information provided by the emotion engine. For example, if the server recognizes that the user is feeling stressed, it generates a menu using ingredients that have a relaxing effect. The generated menu is sent from the server to the user's device.

[1867] Generate and order materials lists

[1868] The terminal displays a list of ingredients required based on the received menu (e.g., "300g chicken, 2 tomatoes, salad vegetables, 2 cups brown rice") to the user. The user can check and edit this list, and an order is placed using the online supermarket's API. The online supermarket receives the order and delivers it at the specified date and time.

[1869] QR Code Generation and Distribution

[1870] Once the order details are confirmed, the server converts the information into a QR code and sends it to the user's device. The user then uses the QR code received on their device to confirm receipt at the online supermarket.

[1871] Save ...

Claims

1. a means for inputting and storing personal information, including health information; A means for inputting and storing information on food ingredients in the home; A means for generating a meal menu using the stored personal information and ingredient information; means for displaying a list of ingredients required based on the generated meal menu; A way to order ingredients via the online supermarket's API, A means of converting order details into a QR code and distributing it; A means for storing purchased ingredient information and providing cooking instructions; The system includes a means for recording information on leftover ingredients and reflecting this information in the next menu generation.

2. 2. The system according to claim 1, wherein menus for multiple days are generated and displayed at once based on the stored personal information and ingredient information.

3. The system according to claim 1, wherein the cooking method is provided via an API of a cooking app, and specific instructions or a link to a cooking video are provided.

Images