System

Abstract

A system is provided.SOLUTION: A system comprising: means for causing a user to input stock information of a home refrigerator, a family structure, and information of cookware possessed by the user; means for storing the input information in a database; means for acquiring bargain information from a nearby store; means for proposing a recipe generated based on the stored information and the acquired bargain information; and means for transmitting the proposed recipe to a terminal of the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] For many households, daily meal planning is a burden. It is also difficult to find recipes that fit each household's needs and resources. Furthermore, it is also a challenge to purchase economically efficient ingredients while taking into account sales information and reducing food waste. Furthermore, there is a demand to reduce the cooking effort and burden at home. This requires centralized management of information that differs from household to household and to suggest optimal recipes and cooking methods. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for a user to input information about the inventory of a household refrigerator, family composition, and cooking equipment owned by the user; a means for saving the input information in a database; a means for acquiring sales information from nearby stores; a means for generating and suggesting recipes based on the saved information and the acquired sales information; and a means for transmitting the suggested recipes to the user's device. The system also includes a means for sending the suggested recipes to a cooking robot and providing cooking instructions, thereby reducing the effort required for cooking at home. The system also builds a community platform for users to share leftover ingredients and cooking equipment, thereby reducing food waste and efficiently utilizing necessary resources. This dynamically optimizes meal planning according to each household's needs and resources, improving QoL.

[0006] "User" refers to an individual household member who utilizes the system to input household information and receive recipe suggestions.

[0007] "Refrigerator inventory information" refers to information about the types and quantities of ingredients and foods currently stored in the user's home.

[0008] "Family composition" refers to information about the number of people living in the user's household and each person's food preferences and allergies.

[0009] "Cooking equipment" refers to tools and home appliances used for cooking that are present in the user's home.

[0010] "Database" refers to a repository of information that stores collected user and special offers information and makes it accessible when needed.

[0011] "Special sale information" refers to discount information on ingredients and food products at nearby stores.

[0012] A "generated recipe" refers to a cooking procedure and a list of ingredients required for the recipe that are generated based on the user's refrigerator inventory information, family composition, cooking utensils, and special sale information.

[0013] "Terminal" refers to the device a user uses to access the system, enter information, and receive suggested recipes.

[0014] A "cooking robot" is a device that automatically cooks food and has the ability to cook on behalf of the user after receiving instructions from the system.

[0015] A "community platform" refers to an online information exchange forum where users can share leftover ingredients and cooking utensils.

[0016] "Food waste" refers to ingredients and food that are discarded without being consumed, and one of the aims of the system is to reduce this.

[0017] "QoL" stands for Quality of Life and refers to quality of life and satisfaction. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] This invention is a system that collects household information from users and generates optimal recipes based on that information. This system operates in cooperation with three entities: a server, a terminal, and the user.

[0040] Entering and collecting user information

[0041] User

[0042] Users enter information about their refrigerator inventory, household composition, and cooking appliances through an application or web portal. This information is stored on the user's device and then sent to the server.

[0043] Terminal

[0044] After receiving the information entered by the user, the device sends the data to the server, including the types and quantities of ingredients in the refrigerator, the number of family members and their food preferences, and the cooking equipment installed in the home.

[0045] server

[0046] The server stores the information sent from the device in a database that stores detailed household information for each user.

[0047] Collecting special sale information

[0048] server

[0049] The server periodically collects sales information by calling the sales information API in cooperation with nearby stores. The sales information of the stores is obtained from this API and saved in a database. This allows the latest sales information to be reflected in real time.

[0050] Recipe Generation

[0051] server

[0052] The server retrieves the user's household information and sales information from the database, and uses generative AI to generate recipes based on this information. The generated recipes include optimal cooking steps and ingredient lists that take into account the current refrigerator inventory and sales information.

[0053] Recipe presentation and selection

[0054] server

[0055] The server sends the generated recipe to the user's device and presents several recipe suggestions so that the user can choose from multiple recipes.

[0056] Terminal

[0057] The terminal displays the recipe sent from the server and offers the user options.

[0058] User

[0059] The user selects the recipe that best suits their taste from the suggested recipes, and the selected information is sent back to the server via the terminal.

[0060] Automated cooking and delivery services

[0061] server

[0062] The server sends the recipe selected by the user to the cooking robot and provides cooking instructions, and the cooking robot automatically starts cooking based on the instructions sent.

[0063] cooking robot

[0064] The cooking robot automatically cooks according to the recipe received from the server and serves the finished dish to the user, or works with a delivery service to deliver it to the user.

[0065] Community Collaboration

[0066] User

[0067] Users can share their leftover ingredients and cooking utensils with other users through the community platform, which helps reduce food waste and makes efficient use of the ingredients and cooking utensils they need.

[0068] server

[0069] The server manages transaction information between users and performs appropriate matching. This information is stored in a database and is reflected in the next recipe suggestions.

[0070] Specific examples

[0071] For example, suppose User A has 6 eggs, 500ml of milk, 3 carrots, and 200g of chicken in his refrigerator. User A also inputs that he has a family of 3 people, some of whom are vegetarian and some of whom have low-carb diets. He also owns an oven, a microwave, and a blender.

[0072] The server records in its database the sale information it has obtained from nearby stores, such as tomatoes at 100 yen per bag and beef at 500 yen per kg.

[0073] Based on this information, the server uses a generative AI to generate the following recipe:

[0074] Vegetable Omelette: Steps to make an omelet in a frying pan using eggs, carrots, and tomatoes.

[0075] Chicken and Tomato Salad: Steps for a salad made with boiled chicken, tomatoes, and lettuce.

[0076] The user selects a vegetable omelet from the list, and the server sends this information to the cooking robot, which then automatically cooks the omelet and serves it to the user at the scheduled time or delivers it via a delivery service.

[0077] In this way, the system of the present invention can make the most of each household's needs and resources to provide an efficient and satisfying dining experience.

[0078] The processing flow will be explained below.

[0079] Step 1:

[0080] A user accesses an application or web portal and enters information about the inventory of their home refrigerator, their family members, and the cooking appliances they own.

[0081] Step 2:

[0082] The terminal receives user input information, formats it, and sends it to the server.

[0083] Step 3:

[0084] The server stores the user information received from the device in a database, which includes refrigerator inventory information, family composition, and a list of cooking utensils.

[0085] Step 4:

[0086] The server calls the sale information API to get sale information from nearby stores. The sale information includes the store name, sale items, and prices.

[0087] Step 5:

[0088] The server stores the acquired sale information in a database.

[0089] Step 6:

[0090] The server retrieves the saved user information and sale information from the database and passes it to the generation AI engine.

[0091] Step 7:

[0092] The generative AI engine generates optimal recipes based on the acquired data, including a list of ingredients and cooking instructions.

[0093] Step 8:

[0094] The server sends the list of generated recipes to the user's terminal.

[0095] Step 9:

[0096] The user checks the recipes sent and selects the recipe they like.

[0097] Step 10:

[0098] The terminal again transmits the user's selection information to the server.

[0099] Step 11:

[0100] The server sends the selected recipe to the cooking robot and instructs it to start cooking.

[0101] Step 12:

[0102] The cooking robot automatically starts cooking according to the recipe information received from the server.

[0103] Step 13:

[0104] After the cooking robot completes the cooking, it sends a cooking completion notification to the server.

[0105] Step 14:

[0106] The server receives the cooking completion notification and issues delivery instructions to the delivery service as needed.

[0107] Step 15:

[0108] A delivery service delivers pre-cooked meals to a user's location.

[0109] Step 16:

[0110] Users share leftover ingredients and cooking utensils with other users through a community platform.

[0111] Step 17:

[0112] The server stores the information posted on the community platform in a database and uses it the next time a recipe is generated.

[0113] This process dynamically generates optimal recipes based on the user's household information and sales information, and in cooperation with the cooking robot, achieves hassle-free cooking. Furthermore, the community platform also promotes the effective use of resources.

[0114] Example 1

[0115] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0116] Traditionally, when cooking at home, users had to come up with their own recipes based on the ingredients and cooking utensils they had, which was time-consuming and labor-intensive. Furthermore, there was a lack of mechanisms for efficient shopping using sales information or for sharing ingredients and cooking utensils with other users. To solve these issues, a system that provides an efficient and satisfying dining experience is needed.

[0117] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0118] In this invention, the server includes means for allowing users to input information about their home refrigerator inventory, family composition, and cooking utensils; means for saving the input information in a database; means for acquiring sales information from nearby stores; means for generating and suggesting recipes using a generative AI model; and means for transmitting the suggested recipes to the user's device. This allows users to easily obtain optimal recipes based on the ingredients, cooking utensils, and sales information available at home, enabling them to cook efficiently and save time and effort. Furthermore, sharing ingredients and cooking utensils among users promotes the reduction of food waste and the efficient use of resources.

[0119] A "user" is an entity that uses the system to provide refrigerator inventory information, family composition, and cooking utensil information.

[0120] A "database" is a storage device for storing and managing information entered by users, sale information, and the like.

[0121] "Special sale information" is information obtained from nearby stores about products being sold at discounted prices.

[0122] A "generative AI model" is an artificial intelligence algorithm that generates recipes based on accumulated data.

[0123] "Suggestion" is the act of showing a recipe generated by a generative AI model to a user.

[0124] A "terminal" is an electronic device through which a user inputs information and receives suggested recipes.

[0125] A "cooking device" is a machine that automatically cooks food based on a recipe sent from a server.

[0126] A "community platform" is an online gathering place where users can share leftover ingredients and cooking utensils.

[0127] This invention is a system that collects household information from users and generates optimal recipes based on that information. This system operates in cooperation with three entities: a server, a terminal, and the user.

[0128] Entering and collecting user information

[0129] User

[0130] Users enter information about their refrigerator inventory, household composition, and cooking appliances through an application or web portal. This information is stored on the user's device and then sent to the server.

[0131] Terminal

[0132] After receiving the information entered by the user, the device sends the data to the server, including the types and quantities of ingredients in the refrigerator, the number of family members and their food preferences, and the cooking equipment installed in the home.

[0133] server

[0134] The server stores the information sent from the device in a database that stores detailed household information for each user.

[0135] Collecting special sale information

[0136] server

[0137] The server periodically collects sales information by calling the sales information API in cooperation with nearby stores. The sales information of the stores is obtained from this API and saved in a database. This allows the latest sales information to be reflected in real time.

[0138] Recipe Generation

[0139] server

[0140] The server retrieves the user's household information and sales information from the database, and then uses the generative AI model to generate recipes based on this information. The generated recipes include optimal cooking steps and ingredient lists that take into account the current refrigerator inventory and sales information.

[0141] Recipe presentation and selection

[0142] server

[0143] The server sends the generated recipe to the user's device and presents several recipe suggestions so that the user can choose from multiple recipes.

[0144] Terminal

[0145] The terminal displays the recipe sent from the server and offers the user options.

[0146] User

[0147] The user selects the recipe that best suits their taste from the suggested recipes, and the selected information is sent back to the server via the terminal.

[0148] Automated cooking and delivery services

[0149] server

[0150] The server sends the recipe selected by the user to the cooking device and provides cooking instructions, and the cooking device automatically starts cooking based on the transmitted instructions.

[0151] cooking equipment

[0152] The cooking device automatically cooks the food according to the recipe received from the server and serves the finished dish to the user, or delivers it to the user in cooperation with a delivery service.

[0153] Community Collaboration

[0154] User

[0155] Users can share their leftover ingredients and cooking utensils with other users through the community platform, which helps reduce food waste and makes efficient use of the ingredients and cooking utensils they need.

[0156] server

[0157] The server manages transaction information between users and performs appropriate matching. This information is stored in a database and is reflected in the next recipe suggestions.

[0158] Specific examples

[0159] For example, suppose User A has 6 eggs, 500ml of milk, 3 carrots, and 200g of chicken in his refrigerator. User A also inputs that he has a family of 3 people, some of whom are vegetarian and some of whom have low-carb diets. He also owns an oven, a microwave, and a blender.

[0160] The server records in its database the sale information it has obtained from nearby stores, such as tomatoes at 100 yen per bag and beef at 500 yen per kg.

[0161] With this information, the server uses a generative AI model to generate the following recipe:

[0162] Vegetable Omelet: How to make an omelet in a frying pan using eggs, carrots, and tomatoes

[0163] Chicken and Tomato Salad: A salad made with boiled chicken, tomatoes, and lettuce.

[0164] The user selects the vegetable omelet, and the server sends this information to the cooking device, which then automatically cooks the omelet and serves it to the user at the scheduled time or delivers it via a delivery service.

[0165] In this way, the system of the present invention can make the most of each household's needs and resources to provide an efficient and satisfying dining experience.

[0166] Prompt Sentence Examples

[0167] "In the refrigerator, there are six eggs, 500ml of milk, three carrots, and 200g of chicken. There are three people in my family, and some are vegetarian and some are low-carb. I also have an oven, microwave, and blender. I'm currently seeing special offers on tomatoes for 100 yen per bag and beef for 500 yen per kg. Please generate the best recipe for me."

[0168] By inputting this prompt into a generative AI model, the optimal recipe based on the user's needs can be generated.

[0169] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0170] Step 1:

[0171] Entering user information

[0172] Users open an application or web portal and enter information about their refrigerator inventory, household composition, and cooking appliances, which is temporarily stored on the user's device.

[0173] Input: Ingredients in the refrigerator (e.g., 6 eggs, 500 ml of milk, 3 carrots, 200 g of chicken), family size (e.g., 3 people), cooking equipment (e.g., oven, microwave, blender)

[0174] Output: Temporarily stores the household information entered by the user.

[0175] Step 2:

[0176] Sending user information

[0177] The terminal transmits the information input by the user to the server. Specifically, the information is transmitted when the send button is clicked.

[0178] Input: Home information entered by the user

[0179] Output: Home information received by the server

[0180] Step 3:

[0181] Saving user information

[0182] The server stores the home information sent from the device in a database, and the stored data is organized by user.

[0183] Input: Home information sent from the device

[0184] Output: Household information stored in a database

[0185] Step 4:

[0186] Collecting special sale information

[0187] The server periodically calls the sales information API to obtain the latest sales information from nearby stores, and stores this information in a database.

[0188] Input: Call the sale information API

[0189] Output: Special sale information stored in the database

[0190] Step 5:

[0191] Recipe Generation

[0192] The server retrieves the user's household information and special sale information from the database and inputs it as a prompt into the generative AI model, which then generates the optimal recipe based on that information.

[0193] Input: Household information and special sale information obtained from the database

[0194] Output: Recipes generated by the generative AI model

[0195] Specific operation: The server generates a prompt (e.g., "In the refrigerator, there are 6 eggs, 500ml of milk, 3 carrots, and 200g of chicken. There are 3 people in the family, and some prefer vegetarian and low-carb foods. There is also an oven, microwave, and blender. There are special offers on tomatoes for 100 yen per bag and beef for 500 yen per kg. Please generate the best recipe.") and sends it to the generative AI model.

[0196] Step 6:

[0197] Submitting a Recipe

[0198] The server transmits the generated recipe to the user's terminal.

[0199] Input: Recipes generated by a generative AI model

[0200] Output: Recipe sent to terminal

[0201] Step 7:

[0202] Viewing Recipes

[0203] The terminal displays the recipe sent from the server to the user.

[0204] Input: Recipe sent from the server

[0205] Output: The recipe that is displayed to the user

[0206] Specific operation: Recipes for "Vegetable Omelette" and "Chicken and Tomato Salad" displayed on the application screen

[0207] Step 8:

[0208] Recipe Selection

[0209] The user selects a recipe from the displayed list, and the selection information is sent to the server via the terminal.

[0210] Input: User selected recipe

[0211] Output: Selected recipe information sent to the server

[0212] Step 9:

[0213] Cooking Instructions

[0214] The server transmits the recipe selected by the user to the cooking device and issues cooking instructions.

[0215] Input: Selected recipe information

[0216] Output: Cooking instructions sent to the cooking device

[0217] Step 10:

[0218] Automatic cooking

[0219] The cooking device automatically starts cooking based on the recipe information received from the server, and once the cooking is complete, it serves the food to the user or delivers it via a delivery service.

[0220] Input: Cooking instructions from the server

[0221] Output: Finished dish

[0222] Specific operation: The cooking device automatically cooks an omelet and serves the finished product to the user.

[0223] Step 11:

[0224] Community Collaboration

[0225] Users share their leftover ingredients and cooking utensils with other users through the community platform, and this sharing information is sent to the server.

[0226] Input: Information on leftover ingredients and cooking utensils shared by users

[0227] Output: Shared information sent to the server

[0228] Step 12:

[0229] Management of transaction information

[0230] The server manages transaction information between users and performs matching as appropriate. Matching information is stored in a database and reflected in the next recipe suggestions.

[0231] Input: Shared ingredients and cooking utensil information

[0232] Output: Transaction information stored in the database

[0233] Specific operation: Matching transaction information between users such as "I have two eggs left" or "I need two eggs"

[0234] Through these steps, the system generates and provides optimal recipes tailored to the needs of each household, providing an efficient and satisfying dining experience.

[0235] (Application example 1)

[0236] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0237] Conventional recipe suggestion systems were able to generate recipes based on information about the household refrigerator's inventory, household configuration, and cooking utensils, but they lacked support for users to have an efficient and effective shopping experience in a physical store. They also lacked the functionality to instantly show the generated recipe to the user and provide a shopping list on the spot. This made it difficult for users to smoothly shop and reduce the effort required for cooking.

[0238] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0239] In this invention, the server includes a means for allowing a user to input information about the inventory of a home refrigerator, family composition, and cooking equipment owned by the user, a means for storing the input information in a database, and a means for acquiring sales information from nearby stores. This allows recipes to be displayed using a smart device in a physical store. Furthermore, by adding a means for generating a shopping list based on the recipe and guiding the user through shopping in the store, the user can shop efficiently and reduce the effort required for cooking.

[0240] "Means for allowing users to input information on the inventory of their household refrigerators, their family composition, and the cooking equipment they own" refers to a method in which users input the ingredients, number of people in their household, and the cooking equipment they can use, and collect that information.

[0241] The term "means for storing the input information in a database" refers to a system or device for electronically storing and managing information input by a user.

[0242] "Means for obtaining sale information from nearby stores" refers to a method or system for obtaining sale information at a store in cooperation with the store.

[0243] "Means for suggesting recipes generated based on the stored information and the acquired sale information" refers to a method or system for generating and suggesting appropriate recipes based on stored household information and sale information.

[0244] "Means for sending the proposed recipe to a user's terminal" refers to a method or system for sending the generated recipe to an electronic terminal, such as a smartphone or tablet.

[0245] "Means for displaying the recipe using a smart device in a physical store" refers to a method or system for displaying a recipe using a smart device while the user is in the physical store.

[0246] "Means for generating a shopping list based on the recipe and guiding shopping in the store" refers to a method or system for listing ingredients required for the generated recipe and allowing efficient shopping in the store based on that list.

[0247] The present invention is a system that collects household information from users and generates optimal recipes based on that information. This system operates in cooperation with three entities: a server, a terminal, and a user.

[0248] Entering and collecting user information

[0249] User

[0250] Users input information about their home refrigerator inventory, family composition, and cooking utensils via smart devices such as smartphones, tablets, smart glasses, etc. This information is temporarily stored on the user's device and then sent to the server.

[0251] Terminal

[0252] After receiving the information entered by the user, the device sends the data to the server, including the types and quantities of ingredients in the refrigerator, the number of family members and their food preferences, and the cooking equipment installed in the home.

[0253] server

[0254] The server stores the information sent from the device in a database that stores detailed household information for each user.

[0255] Collecting special sale information

[0256] The server periodically collects sales information by calling the sales information API in cooperation with nearby stores. The sales information of the stores is obtained from this API and saved in a database. This allows the latest sales information to be reflected in real time.

[0257] Recipe Generation

[0258] The server retrieves the user's household information and sales information from the database, and then uses generative AI (e.g., OpenAI's GPT-3 model) to generate a recipe based on this information. The generated recipe includes optimal cooking steps and a list of ingredients, taking into account the current refrigerator inventory and sales information.

[0259] Recipe presentation and selection

[0260] The server sends the generated recipe to the user's smart device terminal. It presents several recipe suggestions so that the user can choose from multiple recipes. The terminal displays the recipes sent from the server and offers options to the user. The user selects the recipe that best suits their preferences from the suggested recipes. The selected information is sent back to the server via the terminal.

[0261] Displaying recipes and generating shopping lists in physical stores

[0262] The user can view the recipes using a smart device in a physical store. The smart device displays the recipes based on the user's household information and sales information. The smart device also generates a shopping list based on the generated recipes, guiding the user to efficiently shop in the store.

[0263] For example, suppose User A has 6 eggs, 500ml of milk, 3 carrots, and 200g of chicken in his refrigerator. User A also inputs that he has a family of 3 people and owns an oven, microwave, and blender. The server uses a generation AI based on sale information obtained from nearby stores to generate the following recipe:

[0264] "Vegetable Omelette": Steps for making an omelet in a frying pan using eggs, carrots, and tomatoes.

[0265] "Chicken and Tomato Salad": Steps for making a salad of boiled chicken, tomatoes, and lettuce.

[0266] An example of a generated prompt is:

[0267] "The refrigerator contains {'eggs': 6, 'milk': '500ml', 'carrot': 3, 'chicken': '200g'}. There are three people in the family. The kitchen appliances are ['oven', 'microwave', 'blender']. The sale information is ['tomatoes: 100 yen / bag', 'beef: 500 yen / kg']. Please suggest a recipe based on this."

[0268] Users can view and select recipes using their smart devices in a physical store. Based on their selection, the system generates a shopping list with the ingredients they need, allowing users to shop efficiently in the store and reduce the effort required for cooking. In this way, the system of the present invention can maximize the use of each household's needs and resources to provide an efficient and satisfying dining experience.

[0269] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0270] Step 1:

[0271] A user uses a smart device (such as a smartphone, tablet, or smart glasses) to input information about the inventory of their home refrigerator, their family composition, and the cooking appliances they own. This input information is temporarily stored in the smart device's application.

[0272] Input: refrigerator inventory information, family composition, cooking utensil information

[0273] Output: Temporary data storage (smart device)

[0274] Step 2:

[0275] The device receives the information entered by the user and transmits the data to the server, including the types and quantities of ingredients in the refrigerator, the number of family members and their food preferences, and the cooking utensils available in the home.

[0276] Input: Home information entered by the user

[0277] Output: Send data to the server

[0278] Step 3:

[0279] The server stores the information sent from the devices in a database that stores detailed household information for each individual user.

[0280] Input: Home information sent from the device

[0281] Output: Save data to database

[0282] Step 4:

[0283] The server periodically collects sales information from nearby stores and calls the sales information API. This API retrieves sales information from the stores in real time and stores it in a database.

[0284] Input: Data retrieval request from the special sale information API

[0285] Output: Save the acquired sales information in a database

[0286] Step 5:

[0287] The server retrieves the user's household information and special sale information from the database, and then generates a recipe using a generative AI model (e.g., OpenAI's GPT-3). A prompt is entered into the generative AI model to generate an appropriate recipe. An example of a prompt is as follows:

[0288] "The refrigerator contains {'eggs': 6, 'milk': '500ml', 'carrot': 3, 'chicken': '200g'}. There are three people in the family. The kitchen appliances are ['oven', 'microwave', 'blender']. The sale information is ['tomatoes: 100 yen / bag', 'beef: 500 yen / kg']. Please suggest a recipe based on this."

[0289] Input: User's home information, special offer information, prompt text

[0290] Output: Generated recipe list

[0291] Step 6:

[0292] The server sends the generated recipe to the user's smart device, where several recipes are presented to the user so that the user can choose from them.

[0293] Input: Generated recipe list

[0294] Output: Send recipe to smart device

[0295] Step 7:

[0296] The user checks the recipes presented on the smart device and selects the recipe that suits their taste. The selected recipe information is then sent back to the server via the device.

[0297] Input: Multiple recipe suggestions, user selection

[0298] Output: Selected recipe information sent to server via terminal

[0299] Step 8:

[0300] The user displays the selected recipe using a smart device in a physical store, and the smart device generates a shopping list of ingredients based on the selected recipe, helping the user to shop efficiently in the store.

[0301] Input: User selected recipe

[0302] Output: Shopping list generation and in-store shopping guidance

[0303] The above steps enable optimal recipe suggestions based on household information and special sale information, as well as efficient shopping in physical stores.

[0304] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0305] This system generates optimal recipes by taking into account the user's household information and emotional state, and performs automatic cooking in cooperation with a cooking robot. This system functions through the cooperation of four entities: a server, a terminal, a user, and an emotion engine.

[0306] Entering and collecting user information

[0307] User

[0308] Users enter information about their refrigerator inventory, household composition, and cooking appliances through a dedicated application or web portal. This information is stored on the user's device and then sent to the server.

[0309] Terminal

[0310] The device receives information entered by the user, formats it, and sends it to a server. This data includes the types and quantities of food in the refrigerator, the number of family members and their food preferences, and the cooking equipment in the home.

[0311] server

[0312] The server stores the user information sent from the terminal in a database, which stores detailed household information for each user.

[0313] Collecting special sale information

[0314] server

[0315] The server periodically collects sales information by calling the sales information API in cooperation with nearby stores. The sales information of the stores is obtained from this API and saved in a database. This allows the latest sales information to be reflected in real time.

[0316] Recipe generation and emotional engine utilization

[0317] server

[0318] The server retrieves the user information and special sale information stored in the database and passes them to the generative AI engine, which then generates the optimal recipe. The generated recipe includes a list of ingredients and cooking instructions.

[0319] Emotion Engine

[0320] The emotion engine recognizes the user's emotional state in real time and transmits the emotion data to the server, which reflects the user's detailed emotional state, such as whether they are relaxed or stressed.

[0321] server

[0322] The server optimizes the generated recipes based on the emotion data sent from the emotion engine. For example, if the user is feeling stressed, it will prioritize recipes that are quick and easy to prepare.

[0323] Recipe presentation and selection

[0324] server

[0325] The server sends the optimized recipe to the user's device and presents several options so that the user can choose from multiple recipes.

[0326] Terminal

[0327] The terminal displays the recipe sent from the server and offers the user options.

[0328] User

[0329] The user selects the recipe that best suits their taste from the suggested recipes, and the selected information is sent back to the server via the terminal.

[0330] Automated cooking and delivery services

[0331] server

[0332] The server sends the recipe selected by the user to the cooking robot and instructs it to start cooking. The cooking robot automatically starts cooking based on the instructions sent.

[0333] cooking robot

[0334] The cooking robot automatically cooks according to the recipe information received from the server, and after cooking is complete, it sends a cooking completion notification to the server.

[0335] server

[0336] The server receives the cooking completion notification and issues delivery instructions to the delivery service as needed.

[0337] Community Collaboration

[0338] User

[0339] Users can share their leftover ingredients and cooking utensils with other users through the community platform, which helps reduce food waste and makes efficient use of the ingredients and cooking utensils they need.

[0340] server

[0341] The server manages transaction information between users and performs matching as appropriate. This information is saved in a database and used the next time a recipe is generated.

[0342] Specific examples

[0343] For example, suppose User B has six eggs, 500ml of milk, three carrots, and 200g of chicken in his refrigerator. User B also inputs that he has three family members, some of whom are vegetarian and some of whom prefer low-carb meals. Furthermore, suppose he owns an oven, a microwave, and a blender. If User B's emotional state indicates "stress," the server will optimize the recipe suggestions based on this emotional data.

[0344] The server has recorded in its database sales information obtained from nearby stores that tomatoes are 100 yen per bag and beef is 500 yen per kg. Based on this information, the server uses generative AI to generate the following recipe:

[0345] Vegetable Omelette: Steps to make an omelet in a frying pan using eggs, carrots, and tomatoes.

[0346] Chicken and Tomato Salad: Steps for a salad made with boiled chicken, tomatoes, and lettuce.

[0347] Furthermore, taking into account User B's stress level, the server prioritizes an "omelette with lots of vegetables," which can be cooked in a short time. User B selects this recipe, and the server sends instructions to the cooking robot. The cooking robot automatically cooks the omelette based on these instructions and serves it to User B at the set time, or delivers it via a delivery service.

[0348] In this way, the system of the present invention can provide an efficient and satisfying dining experience by taking into maximum consideration the needs and resources of each household, as well as the user's emotional state, thereby improving QoL.

[0349] The processing flow will be explained below.

[0350] Step 1:

[0351] A user accesses an application or web portal and enters information about the household refrigerator inventory, household composition, and cooking equipment they own, such as six eggs, 500ml of milk, three carrots, and 200g of chicken.

[0352] Step 2:

[0353] The terminal receives user-entered information, formats it, and sends it to the server. Specifically, the entered information is converted into JSON format data and sent to the server using the HTTPS protocol.

[0354] Step 3:

[0355] The server stores the user information received from the device in a database, including refrigerator inventory, family composition (e.g., three people in a family, vegetarian, low-carb preference), and a list of cooking appliances (oven, microwave, blender).

[0356] sql

[0357] INSERT INTO user_data (user_id, refrigerator, family, tools)

[0358] VALUES ("user123", '{"eggs":6,"milk":"500ml","carrots":3,"chicken":"200g"}', '{"members":3,"preferences":["vegetarian","low-carb"]}', '["oven","microwave","blender"]');

[0359] Step 4:

[0360] The server calls the special sale information API to get special sale information from nearby stores, such as tomatoes for 100 yen per bag and beef for 500 yen per kg.

[0361] json

[0362] {

[0363] "special_offers": {

[0364] "tomato": "100yen / bag",

[0365] "beef": "500yen / kg"

[0366] }

[0367] }

[0368] Step 5:

[0369] The server stores the acquired sale information in a database.

[0370] sql

[0371] INSERT INTO special_offers (store_id, item, price)

[0372] VALUES ("store456", "tomato", "100yen / bag"), ("store456", "beef", "500yen / kg");

[0373] Step 6:

[0374] The server retrieves the saved user information and sale information from the database and passes it to the generation AI engine.

[0375] sql

[0376] SELECT FROM user_data WHERE user_id = "user123";

[0377] SELECT FROM special_offers WHERE store_id = "store456";

[0378] Step 7:

[0379] The generative AI engine generates optimal recipes based on the acquired data. The generated recipes include a list of ingredients and cooking instructions. For example, it generates recipes for a "vegetable omelet" and a "chicken and tomato salad."

[0380] json

[0381] {

[0382] "recipes": [

[0383] {

[0384] "name": "Vegetable Omelette",

[0385] "ingredients": ["eggs", "carrots", "tomato"],

[0386] "instructions": "Mix eggs, chop vegetables, and fry in a frying pan. Finally, pour in the eggs to make an omelet."

[0387] },

[0388] {

[0389] "name": "Chicken and Tomato Salad",

[0390] "ingredients": ["chicken", "tomato"],

[0391] "instructions": "Boil the chicken and serve it as a salad with tomatoes."

[0392] }

[0393] ]

[0394] }

[0395] Step 8:

[0396] The emotion engine recognizes the user's emotional state in real time and sends the emotion data to the server. For example, if the user is feeling stressed, the information is sent to the server.

[0397] Step 9:

[0398] The server optimizes the generated recipes based on the emotional data from the emotion engine. For example, if the user is feeling stressed, it will prioritize suggesting an "omelette with lots of vegetables," which can be cooked quickly and easily.

[0399] Step 10:

[0400] The server sends a list of optimized recipes to the user's device.

[0401] Step 11:

[0402] The device displays the recipes and offers options to the user, who can then select the recipe that best suits their taste.

[0403] Step 12:

[0404] The recipe information selected by the user is again transmitted to the server via the terminal.

[0405] Step 13:

[0406] The server sends the selected recipe to the cooking robot and instructs it to start cooking.

[0407] Step 14:

[0408] The cooking robot automatically starts cooking according to the recipe information received from the server.

[0409] Step 15:

[0410] After the cooking robot completes the cooking, it sends a cooking completion notification to the server.

[0411] Step 16:

[0412] The server receives the cooking completion notification and issues delivery instructions to the delivery service as needed.

[0413] Step 17:

[0414] A delivery service delivers pre-cooked meals to a user's location.

[0415] Step 18:

[0416] Users share leftover ingredients and cooking utensils with other users through a community platform.

[0417] Step 19:

[0418] The server stores the information posted on the community platform in a database and uses it the next time a recipe is generated.

[0419] This process dynamically and efficiently generates optimal recipes that take into account the user's household information, sales information, and emotional state, and works in conjunction with a cooking robot to achieve hassle-free cooking. Furthermore, the community platform also promotes efficient resource utilization.

[0420] Example 2

[0421] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0422] In modern households, managing refrigerator inventory and planning meals requires time and effort, while food waste is becoming a serious problem. Furthermore, recipe suggestions that take into account the user's emotional state may not be adequately tailored to the user's needs and resources. Therefore, there is a need for a system that can provide efficient and optimal meal plans that take into account food inventory information, sales information, and the user's emotional state.

[0423] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0424] In this invention, the server includes means for having a user input information on household food inventory, household configuration, and owned cooking utensils, means for organizing the input information and transmitting the input information to the server, means for saving the input information in a database, means for acquiring sales information from nearby stores, means for generating recipes using a generative AI model based on the saved information and the acquired sales information, means for recognizing the user's emotional state in real time and transmitting the data to the server, means for optimizing the generated recipe based on the emotional data, means for transmitting the optimized recipe to the user's terminal, and means for the user to select a recipe. This enables optimal recipe suggestions and automatic cooking based on the user's household information and emotional state, thereby reducing food waste and improving user satisfaction.

[0425] "Food inventory information" is data regarding the type, quantity, and condition of food stored in a household refrigerator or pantry.

[0426] "Family composition" is information about the number of family members in the user's household and their individual food preferences.

[0427] "Cooking utensils" is information about the types and number of utensils and equipment used for cooking that the user owns.

[0428] "Special Offers" is data about discounts and specially priced food items offered at nearby stores.

[0429] A "generative AI model" is an artificial intelligence algorithm that generates optimal recipes based on the user's household information and special sale information.

[0430] "Emotional state" is data that assesses a user's current emotions and stress levels in real time.

[0431] "Optimization" is the process of improving the generated recipe by taking into account the user's emotional state and other variables.

[0432] A "server" is a computer system that receives, stores, analyzes input information, and performs recipe generation and optimization.

[0433] A "terminal" is a device that a user uses to enter information, receive recipes, and make selections.

[0434] "Automatic cooking" is the process in which a cooking robot automatically prepares a dish based on a recipe instructed by the server.

[0435] A "community platform" is an online or digital system for users to share leftover ingredients and cooking equipment.

[0436] This system generates optimal recipes by taking into account the user's household information and emotional state, and performs automatic cooking in cooperation with a cooking robot. This system functions in cooperation with multiple entities: a server, a terminal, a user, and an emotion engine.

[0437] Entering and collecting user information

[0438] Users use a dedicated application or web portal to enter information about the food inventory in their refrigerator, their family composition, and the cooking equipment they own. This information is stored on the user's device and then sent to a server. The device then organizes the information entered by the user and sends it to the server. This data includes the types and quantities of ingredients in the refrigerator, the number of family members and their food preferences, and the cooking equipment installed in the home.

[0439] The server stores the user information sent from the device in a database. Detailed household information for each user is stored in the database. The server periodically collects sales information by linking with nearby stores and calling the sales information API. The sales information for each store obtained from this API is stored in the database. This allows the latest sales information to be reflected in real time.

[0440] Recipe generation and emotional engine utilization

[0441] The server retrieves saved user information and special sale information from the database and provides a prompt to the AI ​​engine based on this information. The AI ​​engine then generates the optimal recipe. For example, the prompt is "Please generate a recipe for a family of three that is vegetarian-friendly." The generated recipe includes a list of ingredients and cooking instructions.

[0442] The emotion engine recognizes the user's emotional state in real time and sends that emotional data to the server. This data reflects the user's detailed emotional state, such as whether they are relaxed or stressed. The server optimizes the generated recipes based on the emotional data sent from the emotion engine. For example, if the user is feeling stressed, it will prioritize recipes that are quick and easy to prepare.

[0443] Recipe presentation and selection

[0444] The server sends the optimized recipe to the user's device. The user is presented with several options to choose from. The device displays the recipe sent from the server and provides the user with options.

[0445] The user selects the recipe that best suits their taste from the suggested recipes, and this selection information is sent back to the server via the terminal.

[0446] Automated cooking and delivery services

[0447] The server sends the recipe selected by the user to the cooking robot and instructs it to start cooking. The cooking robot automatically starts cooking based on the instructions sent. After cooking is complete, it sends a cooking completion notification to the server.

[0448] The server receives the cooking completion notification and issues delivery instructions to the delivery service as needed.

[0449] Community Collaboration

[0450] Through the community platform, users can share their surplus ingredients and cooking utensils with other users. This reduces food waste and allows for efficient use of necessary ingredients and cooking utensils. The server manages transaction information between users and performs matching. This information is stored in a database and will be used the next time recipes are generated.

[0451] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0452] Step 1:

[0453] Users use a dedicated application or web portal to enter information about their household food inventory, household composition, and cooking equipment they own. For example, they enter information such as the number of eggs, 500ml of milk, 3 carrots, and 200g of chicken in the refrigerator. The data entered by the user is stored on the device.

[0454] Input: household food inventory information, household composition, cooking utensil information

[0455] Output: User information stored on the device

[0456] Step 2:

[0457] The device receives the information entered by the user, formats it, and sends it to the server. The data is converted to, for example, JSON format and sent to the server. Specific operations include the device organizing the user data, packetizing it in the appropriate format, and sending it to the server via the network.

[0458] Input: User information stored on the device

[0459] Output: User information sent to the server

[0460] Step 3:

[0461] The server receives the user information sent from the device and stores it in a database. Specifically, it parses the received JSON data and stores it in a structured database. This allows detailed household information for each user to be accumulated in the database.

[0462] Input: User information sent to the server

[0463] Output: User information stored in the database

[0464] Step 4:

[0465] The server periodically connects with nearby stores and calls the sale information API to collect sale information. From this API, sale information such as "Tomatoes for 100 yen / bag, beef for 500 yen / kg" is obtained. The obtained information is stored in a database.

[0466] Input: Special sale information obtained from the special sale information API

[0467] Output: Special sale information stored in the database

[0468] Step 5:

[0469] The server retrieves user information and special sale information from the database and provides a prompt to the AI ​​engine. For example, the prompt might be, "Please generate a recipe that is suitable for a family of three and suitable for vegetarians." The AI ​​engine then generates the optimal recipe based on this prompt. The generated recipe includes a list of ingredients and cooking instructions.

[0470] Input: User information and special offer information retrieved from the database, prompt text

[0471] Output: Recipe generated by the generative AI engine

[0472] Step 6:

[0473] The emotion engine recognizes the user's emotional state in real time and sends that data to the server. For example, it determines whether the user is feeling stressed and provides that information to the server. The server then optimizes the generated recipes based on the data sent from the emotion engine. Specifically, it suggests recipes that are quick and easy to prepare to users who are feeling stressed.

[0474] Input: Real-time user emotional state data

[0475] Output: Optimized recipe taking into account emotional state

[0476] Step 7:

[0477] The server generates an optimized recipe and sends it to the user's device. The device displays the received recipe and offers the user several options. For example, it presents recipes such as "vegetable omelet" and "chicken and tomato salad."

[0478] Input: Optimized recipe generated on the server

[0479] Output: Recipe choices displayed on terminal

[0480] Step 8:

[0481] The user uses the terminal to select a desired recipe from the multiple recipes presented, and this selection information is sent back to the server from the terminal.

[0482] Input: User selected recipe

[0483] Output: Selected recipe information sent to the server

[0484] Step 9:

[0485] The server sends the selected recipe information to the cooking robot and instructs it to start cooking. The cooking robot automatically cooks the food based on the instructed recipe. For example, after cooking a "vegetable omelet," it sends a cooking completion notification to the server.

[0486] Input: Selected recipe information

[0487] Output: Automatic cooking by cooking robot, notification of cooking completion

[0488] Step 10:

[0489] The server receives a cooking completion notification from the cooking robot and, based on this, issues delivery instructions to the delivery service as needed. If the user wants to receive the food at home, the server delivers it directly.

[0490] Input: Cooking completion notification

[0491] Output: Delivery instructions to the delivery service, delivery of food to the user

[0492] The above processing steps enable optimal recipe suggestions and automatic cooking that take into account the user's home information and emotional state.

[0493] (Application example 2)

[0494] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0495] In today's busy lifestyles, preparing meals at home is a time-consuming and labor-intensive task that can be stressful for users. It is particularly difficult to select efficient, cost-effective recipes while taking into account the household's ingredient inventory and the family's food preferences. Furthermore, the user's emotional state also influences meal selection, so there is a need for recipe suggestions that reduce stress and promote relaxation. The present invention solves these problems by providing a system that proposes optimal recipes that take the user's emotional state into account and enables automatic cooking.

[0496] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0497] In this invention, the server includes: means for prompting a user to input information about the user's refrigerator inventory, family composition, and cooking utensils; means for saving the input information in a database; means for acquiring sales information from nearby stores; means for suggesting recipes generated using a generative AI model based on the saved information and the acquired sales information; means for acquiring and recognizing the user's emotional state; means for optimizing the generated recipes based on the acquired emotional state; and means for transmitting the suggested recipes to the user's device and providing multiple options. This allows for the creation of recipes that reflect the user's refrigerator inventory and family food preferences, and furthermore, by taking the user's emotional state into consideration, more optimized recipe suggestions are possible. Furthermore, providing multiple options allows for flexible responses to the user's lifestyle and mood at any given time, thereby reducing the effort and stress of cooking.

[0498] A "user" is an entity that uses this system to input information about the inventory of a household refrigerator, family composition, and cooking equipment, and receives optimal recipe suggestions.

[0499] "Home refrigerator inventory information" is information about the types and quantities of food ingredients that a user keeps in their home.

[0500] "Family composition" is information about the members of the user's household, including the number of people living in the household, their age groups, food preferences, and allergy information.

[0501] "Information about cooking utensils owned" is information about the types and quantities of cooking utensils that the user owns in the home.

[0502] The "database" is an electronic information storage device for storing and managing information input by the user and acquired sale information.

[0503] "Special sale information" is information about discounts and special sales on food and ingredients obtained from nearby stores.

[0504] A "generative AI model" is an artificial intelligence algorithm that automatically generates optimal recipes based on user information and special sale information.

[0505] "User's emotional state" is information about the mental state of the user, such as whether the user is relaxed or stressed.

[0506] "Optimizing a recipe" refers to processing the generated recipe according to the user's emotional state to make more appropriate cooking suggestions.

[0507] A "user terminal" is an electronic device, such as a smartphone or tablet, through which a user inputs information and receives suggested recipes.

[0508] A "cooking robot" is a mechanical device that automatically cooks food based on a recipe selected by the user.

[0509] A "community platform" is a shared space on the Internet where users can share and exchange leftover ingredients and cooking utensils.

[0510] This system generates optimal recipes by taking into account the user's household information and emotional state, and performs automatic cooking in cooperation with food delivery and cooking robots. This system functions through the cooperation of four entities: a server, a terminal, a user, and an emotion engine.

[0511] Entering and collecting user information

[0512] User

[0513] Users enter information about their refrigerator inventory, household composition, and cooking appliances through a dedicated application or web portal. This information is stored on the user's device and then sent to the server.

[0514] Terminal

[0515] The device receives information entered by the user, formats it, and sends it to a server. This data includes the types and quantities of food in the refrigerator, the number of family members and their food preferences, and the cooking equipment in the home.

[0516] Collecting special sale information

[0517] server

[0518] The server periodically collects sales information by calling the sales information API in cooperation with nearby stores. The sales information of the stores is obtained from this API and saved in a database. This allows the latest sales information to be reflected in real time.

[0519] Recipe generation and emotional engine utilization

[0520] server

[0521] The server retrieves the user information and special sale information stored in the database and passes them to the generative AI engine, which then generates the optimal recipe. The generated recipe includes a list of ingredients and cooking instructions.

[0522] Emotion Engine

[0523] The emotion engine recognizes the user's emotional state in real time and transmits the emotion data to the server, which reflects the user's detailed emotional state, such as whether they are relaxed or stressed.

[0524] server

[0525] The server optimizes the generated recipes based on the emotion data sent from the emotion engine. For example, if the user is feeling stressed, it will prioritize recipes that are quick and easy to prepare.

[0526] Recipe presentation and selection

[0527] server

[0528] The server sends the optimized recipe to the user's device and presents several options so that the user can choose from multiple recipes.

[0529] Terminal

[0530] The terminal displays the recipe sent from the server and offers the user options.

[0531] User

[0532] The user selects the recipe that best suits their taste from the suggested recipes, and the selected information is sent back to the server via the terminal.

[0533] Automated cooking and delivery services

[0534] server

[0535] The server sends the recipe selected by the user to the cooking robot and instructs it to start cooking. The cooking robot automatically starts cooking based on the instructions sent.

[0536] cooking robot

[0537] The cooking robot automatically cooks according to the recipe information received from the server, and after cooking is complete, it sends a cooking completion notification to the server.

[0538] server

[0539] The server receives the cooking completion notification and issues delivery instructions to the delivery service as needed.

[0540] Community Collaboration

[0541] User

[0542] Users can share their leftover ingredients and cooking utensils with other users through the community platform, which helps reduce food waste and makes efficient use of the ingredients and cooking utensils they need.

[0543] server

[0544] The server manages transaction information between users and performs matching as appropriate. This information is saved in a database and used the next time a recipe is generated.

[0545] Specific examples

[0546] For example, suppose User B has six eggs, 500ml of milk, three carrots, and 200g of chicken in his refrigerator. User B also inputs that he has three family members, some of whom are vegetarian and some of whom prefer low-carb meals. Furthermore, suppose he owns an oven, a microwave, and a blender. If User B's emotional state indicates "stress," the server will optimize the recipe suggestions based on this emotional data.

[0547] The server has recorded in its database sales information obtained from nearby stores that tomatoes are 100 yen per bag and beef is 500 yen per kg. Based on this information, the server uses generative AI to generate the following recipe:

[0548] Vegetable Omelette: Steps to make an omelet in a frying pan using eggs, carrots, and tomatoes.

[0549] Chicken and Tomato Salad: Steps for a salad made with boiled chicken, tomatoes, and lettuce.

[0550] Furthermore, taking into account User B's stress level, the system prioritizes the suggestion of an "omelette with lots of vegetables," which can be cooked in a short time. User B selects this recipe, and the server sends instructions to the cooking robot. The cooking robot automatically cooks the omelette based on these instructions and serves it to User B at the set time, or delivers it via a delivery service. In this way, the system of the present invention provides an efficient and satisfying dining experience, taking into full consideration the needs and resources of each household, as well as the user's emotional state, and thereby improving QoL.

[0551] Example prompt for a generative AI model:

[0552] Generate recipes using the ingredients found in a home refrigerator (6 eggs, 500ml of milk, 3 carrots, 200g of chicken) and cooking equipment (oven, microwave, blender) taking into account the emotional state of stress. Also, reflect sales information (e.g., tomatoes for 100 yen / bag, beef for 500 yen / kg).

[0553] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0554] Step 1:

[0555] Enter and submit user information

[0556] Users enter information about their refrigerator inventory, family composition, and cooking equipment through a dedicated application or web portal. The input data includes specific food information, such as "6 eggs, 500ml of milk, 3 carrots, and 200g of chicken," as well as the number of family members and their food preferences. This information is sent to the device, where it is formatted and then sent to the server.

[0557] Input: Home refrigerator inventory information, family composition, cooking equipment information

[0558] Output: Household information data sent to the server

[0559] Step 2:

[0560] Collecting special sale information

[0561] The server periodically calls the sales information API from nearby stores to collect sales information. Through this API, sales information such as "Tomatoes for 100 yen / bag, beef for 500 yen / kg" is obtained and saved in a database.

[0562] Input: Special sale information obtained from the special sale information API

[0563] Output: Special sale information data stored in a database

[0564] Step 3:

[0565] Obtaining the user's emotional state

[0566] The device uses an emotion engine to recognize the user's emotional state in real time and determines the user's mental state. For example, if the device determines that the user is feeling "stressed," that information is sent to the server.

[0567] Input: User emotion data obtained by the emotion engine

[0568] Output: Emotion data sent to the server

[0569] Step 4:

[0570] Recipe Generation

[0571] The server uses a generative AI model to generate the optimal recipe based on user information and special sale information obtained from the database. It passes a prompt to the generative AI model, generating a prompt such as the following:

[0572] Generate recipes using the ingredients found in a home refrigerator (6 eggs, 500ml of milk, 3 carrots, 200g of chicken) and cooking equipment (oven, microwave, blender) taking into account the emotional state of stress. Also, reflect sales information (e.g., tomatoes for 100 yen / bag, beef for 500 yen / kg).

[0573] Input: User information, sale information, emotion data

[0574] Output: The generated optimal recipe

[0575] Step 5:

[0576] Recipe optimization

[0577] The server optimizes the generated recipes based on the acquired emotional state. For example, if the user is feeling stressed, the server will prioritize the easy-to-make "vegetable omelet" from the multiple generated recipes.

[0578] Input: Generated recipe, emotional state data

[0579] Output: Optimized recipe

[0580] Step 6:

[0581] Recipe presentation and selection

[0582] The server sends the optimized recipe to the terminal and presents multiple options to the user. The user selects the recipe that best suits their preferences and sends the selection back to the server via the terminal.

[0583] Input: Optimized recipe

[0584] Output: The recipe selected by the user

[0585] Step 7:

[0586] Starting automatic cooking

[0587] The server sends the recipe selected by the user to the cooking robot and instructs it to start cooking. The cooking robot automatically starts cooking based on the instructions sent. After cooking is complete, it sends a cooking completion notification to the server. If necessary, it issues delivery instructions to a delivery service.

[0588] Input: User selected recipe

[0589] Output: Food cooked and served

[0590] Step 8:

[0591] Community Collaboration

[0592] The server manages a community platform for users to share leftover ingredients and cooking utensils among themselves. Through the platform, users can share their leftover ingredients and cooking utensils with other users and efficiently use the ingredients and utensils they need.

[0593] Input: Information about leftover ingredients and cooking utensils

[0594] Output: Matching information of shared or exchanged ingredients and cooking utensils

[0595] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0596] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0597] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0598] [Second embodiment]

[0599] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0600] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0601] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0602] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0603] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0604] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0605] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0606] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0607] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0608] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0609] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0610] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0611] This invention is a system that collects household information from users and generates optimal recipes based on that information. This system operates in cooperation with three entities: a server, a terminal, and the user.

[0612] Entering and collecting user information

[0613] User

[0614] Users enter information about their refrigerator inventory, household composition, and cooking appliances through an application or web portal. This information is stored on the user's device and then sent to the server.

[0615] Terminal

[0616] After receiving the information entered by the user, the device sends the data to the server, including the types and quantities of ingredients in the refrigerator, the number of family members and their food preferences, and the cooking equipment installed in the home.

[0617] server

[0618] The server stores the information sent from the device in a database that stores detailed household information for each user.

[0619] Collecting special sale information

[0620] server

[0621] The server periodically collects sales information by calling the sales information API in cooperation with nearby stores. The sales information of the stores is obtained from this API and saved in a database. This allows the latest sales information to be reflected in real time.

[0622] Recipe Generation

[0623] server

[0624] The server retrieves the user's household information and sales information from the database, and uses generative AI to generate recipes based on this information. The generated recipes include optimal cooking steps and ingredient lists that take into account the current refrigerator inventory and sales information.

[0625] Recipe presentation and selection

[0626] server

[0627] The server sends the generated recipe to the user's device and presents several recipe suggestions so that the user can choose from multiple recipes.

[0628] Terminal

[0629] The terminal displays the recipe sent from the server and offers the user options.

[0630] User

[0631] The user selects the recipe that best suits their taste from the suggested recipes, and the selected information is sent back to the server via the terminal.

[0632] Automated cooking and delivery services

[0633] server

[0634] The server sends the recipe selected by the user to the cooking robot and provides cooking instructions, and the cooking robot automatically starts cooking based on the instructions sent.

[0635] cooking robot

[0636] The cooking robot automatically cooks according to the recipe received from the server and serves the finished dish to the user, or works with a delivery service to deliver it to the user.

[0637] Community Collaboration

[0638] User

[0639] Users can share their leftover ingredients and cooking utensils with other users through the community platform, which helps reduce food waste and makes efficient use of the ingredients and cooking utensils they need.

[0640] server

[0641] The server manages transaction information between users and performs appropriate matching. This information is stored in a database and is reflected in the next recipe suggestions.

[0642] Specific examples

[0643] For example, suppose User A has 6 eggs, 500ml of milk, 3 carrots, and 200g of chicken in his refrigerator. User A also inputs that he has a family of 3 people, some of whom are vegetarian and some of whom have low-carb diets. He also owns an oven, a microwave, and a blender.

[0644] The server records in its database the sale information it has obtained from nearby stores, such as tomatoes at 100 yen per bag and beef at 500 yen per kg.

[0645] Based on this information, the server uses a generative AI to generate the following recipe:

[0646] Vegetable Omelette: Steps to make an omelet in a frying pan using eggs, carrots, and tomatoes.

[0647] Chicken and Tomato Salad: Steps for a salad made with boiled chicken, tomatoes, and lettuce.

[0648] The user selects a vegetable omelet from the list, and the server sends this information to the cooking robot, which then automatically cooks the omelet and serves it to the user at the scheduled time or delivers it via a delivery service.

[0649] In this way, the system of the present invention can make the most of each household's needs and resources to provide an efficient and satisfying dining experience.

[0650] The processing flow will be explained below.

[0651] Step 1:

[0652] A user accesses an application or web portal and enters information about the inventory of their home refrigerator, their family members, and the cooking appliances they own.

[0653] Step 2:

[0654] The terminal receives user input information, formats it, and sends it to the server.

[0655] Step 3:

[0656] The server stores the user information received from the device in a database, which includes refrigerator inventory information, family composition, and a list of cooking utensils.

[0657] Step 4:

[0658] The server calls the sale information API to get sale information from nearby stores. The sale information includes the store name, sale items, and prices.

[0659] Step 5:

[0660] The server stores the acquired sale information in a database.

[0661] Step 6:

[0662] The server retrieves the saved user information and sale information from the database and passes it to the generation AI engine.

[0663] Step 7:

[0664] The generative AI engine generates optimal recipes based on the acquired data, including a list of ingredients and cooking instructions.

[0665] Step 8:

[0666] The server sends the list of generated recipes to the user's terminal.

[0667] Step 9:

[0668] The user checks the recipes sent and selects the recipe they like.

[0669] Step 10:

[0670] The terminal again transmits the user's selection information to the server.

[0671] Step 11:

[0672] The server sends the selected recipe to the cooking robot and instructs it to start cooking.

[0673] Step 12:

[0674] The cooking robot automatically starts cooking according to the recipe information received from the server.

[0675] Step 13:

[0676] After the cooking robot completes the cooking, it sends a cooking completion notification to the server.

[0677] Step 14:

[0678] The server receives the cooking completion notification and issues delivery instructions to the delivery service as needed.

[0679] Step 15:

[0680] A delivery service delivers pre-cooked meals to a user's location.

[0681] Step 16:

[0682] Users share leftover ingredients and cooking utensils with other users through a community platform.

[0683] Step 17:

[0684] The server stores the information posted on the community platform in a database and uses it the next time a recipe is generated.

[0685] This process dynamically generates optimal recipes based on the user's household information and sales information, and in cooperation with the cooking robot, achieves hassle-free cooking. Furthermore, the community platform also promotes the effective use of resources.

[0686] Example 1

[0687] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0688] Traditionally, when cooking at home, users had to come up with their own recipes based on the ingredients and cooking utensils they had, which was time-consuming and labor-intensive. Furthermore, there was a lack of mechanisms for efficient shopping using sales information or for sharing ingredients and cooking utensils with other users. To solve these issues, a system that provides an efficient and satisfying dining experience is needed.

[0689] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0690] In this invention, the server includes means for allowing users to input information about their home refrigerator inventory, family composition, and cooking utensils; means for saving the input information in a database; means for acquiring sales information from nearby stores; means for generating and suggesting recipes using a generative AI model; and means for transmitting the suggested recipes to the user's device. This allows users to easily obtain optimal recipes based on the ingredients, cooking utensils, and sales information available at home, enabling them to cook efficiently and save time and effort. Furthermore, sharing ingredients and cooking utensils among users promotes the reduction of food waste and the efficient use of resources.

[0691] A "user" is an entity that uses the system to provide refrigerator inventory information, family composition, and cooking utensil information.

[0692] A "database" is a storage device for storing and managing information entered by users, sale information, and the like.

[0693] "Special sale information" is information obtained from nearby stores about products being sold at discounted prices.

[0694] A "generative AI model" is an artificial intelligence algorithm that generates recipes based on accumulated data.

[0695] "Suggestion" is the act of showing a recipe generated by a generative AI model to a user.

[0696] A "terminal" is an electronic device through which a user inputs information and receives suggested recipes.

[0697] A "cooking device" is a machine that automatically cooks food based on a recipe sent from a server.

[0698] A "community platform" is an online gathering place where users can share leftover ingredients and cooking utensils.

[0699] This invention is a system that collects household information from users and generates optimal recipes based on that information. This system operates in cooperation with three entities: a server, a terminal, and the user.

[0700] Entering and collecting user information

[0701] User

[0702] Users enter information about their refrigerator inventory, household composition, and cooking appliances through an application or web portal. This information is stored on the user's device and then sent to the server.

[0703] Terminal

[0704] After receiving the information entered by the user, the device sends the data to the server, including the types and quantities of ingredients in the refrigerator, the number of family members and their food preferences, and the cooking equipment installed in the home.

[0705] server

[0706] The server stores the information sent from the device in a database that stores detailed household information for each user.

[0707] Collecting special sale information

[0708] server

[0709] The server periodically collects sales information by calling the sales information API in cooperation with nearby stores. The sales information of the stores is obtained from this API and saved in a database. This allows the latest sales information to be reflected in real time.

[0710] Recipe Generation

[0711] server

[0712] The server retrieves the user's household information and sales information from the database, and then uses the generative AI model to generate recipes based on this information. The generated recipes include optimal cooking steps and ingredient lists that take into account the current refrigerator inventory and sales information.

[0713] Recipe presentation and selection

[0714] server

[0715] The server sends the generated recipe to the user's device and presents several recipe suggestions so that the user can choose from multiple recipes.

[0716] Terminal

[0717] The terminal displays the recipe sent from the server and offers the user options.

[0718] User

[0719] The user selects the recipe that best suits their taste from the suggested recipes, and the selected information is sent back to the server via the terminal.

[0720] Automated cooking and delivery services

[0721] server

[0722] The server sends the recipe selected by the user to the cooking device and provides cooking instructions, and the cooking device automatically starts cooking based on the transmitted instructions.

[0723] cooking equipment

[0724] The cooking device automatically cooks the food according to the recipe received from the server and serves the finished dish to the user, or delivers it to the user in cooperation with a delivery service.

[0725] Community Collaboration

[0726] User

[0727] Users can share their leftover ingredients and cooking utensils with other users through the community platform, which helps reduce food waste and makes efficient use of the ingredients and cooking utensils they need.

[0728] server

[0729] The server manages transaction information between users and performs appropriate matching. This information is stored in a database and is reflected in the next recipe suggestions.

[0730] Specific examples

[0731] For example, suppose User A has 6 eggs, 500ml of milk, 3 carrots, and 200g of chicken in his refrigerator. User A also inputs that he has a family of 3 people, some of whom are vegetarian and some of whom have low-carb diets. He also owns an oven, a microwave, and a blender.

[0732] The server records in its database the sale information it has obtained from nearby stores, such as tomatoes at 100 yen per bag and beef at 500 yen per kg.

[0733] With this information, the server uses a generative AI model to generate the following recipe:

[0734] Vegetable Omelet: How to make an omelet in a frying pan using eggs, carrots, and tomatoes

[0735] Chicken and Tomato Salad: A salad made with boiled chicken, tomatoes, and lettuce.

[0736] The user selects the vegetable omelet, and the server sends this information to the cooking device, which then automatically cooks the omelet and serves it to the user at the scheduled time or delivers it via a delivery service.

[0737] In this way, the system of the present invention can make the most of each household's needs and resources to provide an efficient and satisfying dining experience.

[0738] Prompt Sentence Examples

[0739] "In the refrigerator, there are six eggs, 500ml of milk, three carrots, and 200g of chicken. There are three people in my family, and some are vegetarian and some are low-carb. I also have an oven, microwave, and blender. I'm currently seeing special offers on tomatoes for 100 yen per bag and beef for 500 yen per kg. Please generate the best recipe for me."

[0740] By inputting this prompt into a generative AI model, the optimal recipe based on the user's needs can be generated.

[0741] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0742] Step 1:

[0743] Entering user information

[0744] Users open an application or web portal and enter information about their refrigerator inventory, household composition, and cooking appliances, which is temporarily stored on the user's device.

[0745] Input: Ingredients in the refrigerator (e.g., 6 eggs, 500 ml of milk, 3 carrots, 200 g of chicken), family size (e.g., 3 people), cooking equipment (e.g., oven, microwave, blender)

[0746] Output: Temporarily stores the household information entered by the user.

[0747] Step 2:

[0748] Sending user information

[0749] The terminal transmits the information input by the user to the server. Specifically, the information is transmitted when the send button is clicked.

[0750] Input: Home information entered by the user

[0751] Output: Home information received by the server

[0752] Step 3:

[0753] Saving user information

[0754] The server stores the home information sent from the device in a database, and the stored data is organized by user.

[0755] Input: Home information sent from the device

[0756] Output: Household information stored in a database

[0757] Step 4:

[0758] Collecting special sale information

[0759] The server periodically calls the sales information API to obtain the latest sales information from nearby stores, and stores this information in a database.

[0760] Input: Call the sale information API

[0761] Output: Special sale information stored in the database

[0762] Step 5:

[0763] Recipe Generation

[0764] The server retrieves the user's household information and special sale information from the database and inputs it as a prompt into the generative AI model, which then generates the optimal recipe based on that information.

[0765] Input: Household information and special sale information obtained from the database

[0766] Output: Recipes generated by the generative AI model

[0767] Specific operation: The server generates a prompt (e.g., "In the refrigerator, there are 6 eggs, 500ml of milk, 3 carrots, and 200g of chicken. There are 3 people in the family, and some prefer vegetarian and low-carb foods. There is also an oven, microwave, and blender. There are special offers on tomatoes for 100 yen per bag and beef for 500 yen per kg. Please generate the best recipe.") and sends it to the generative AI model.

[0768] Step 6:

[0769] Submitting a Recipe

[0770] The server transmits the generated recipe to the user's terminal.

[0771] Input: Recipes generated by a generative AI model

[0772] Output: Recipe sent to terminal

[0773] Step 7:

[0774] Viewing Recipes

[0775] The terminal displays the recipe sent from the server to the user.

[0776] Input: Recipe sent from the server

[0777] Output: The recipe that is displayed to the user

[0778] Specific operation: Recipes for "Vegetable Omelette" and "Chicken and Tomato Salad" displayed on the application screen

[0779] Step 8:

[0780] Recipe Selection

[0781] The user selects a recipe from the displayed list, and the selection information is sent to the server via the terminal.

[0782] Input: User selected recipe

[0783] Output: Selected recipe information sent to the server

[0784] Step 9:

[0785] Cooking Instructions

[0786] The server transmits the recipe selected by the user to the cooking device and issues cooking instructions.

[0787] Input: Selected recipe information

[0788] Output: Cooking instructions sent to the cooking device

[0789] Step 10:

[0790] Automatic cooking

[0791] The cooking device automatically starts cooking based on the recipe information received from the server, and once the cooking is complete, it serves the food to the user or delivers it via a delivery service.

[0792] Input: Cooking instructions from the server

[0793] Output: Finished dish

[0794] Specific operation: The cooking device automatically cooks an omelet and serves the finished product to the user.

[0795] Step 11:

[0796] Community Collaboration

[0797] Users share their leftover ingredients and cooking utensils with other users through the community platform, and this sharing information is sent to the server.

[0798] Input: Information on leftover ingredients and cooking utensils shared by users

[0799] Output: Shared information sent to the server

[0800] Step 12:

[0801] Management of transaction information

[0802] The server manages transaction information between users and performs matching as appropriate. Matching information is stored in a database and reflected in the next recipe suggestions.

[0803] Input: Shared ingredients and cooking utensil information

[0804] Output: Transaction information stored in the database

[0805] Specific operation: Matching transaction information between users such as "I have two eggs left" or "I need two eggs"

[0806] Through these steps, the system generates and provides optimal recipes tailored to the needs of each household, providing an efficient and satisfying dining experience.

[0807] (Application example 1)

[0808] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0809] Conventional recipe suggestion systems were able to generate recipes based on information about the household refrigerator's inventory, household configuration, and cooking utensils, but they lacked support for users to have an efficient and effective shopping experience in a physical store. They also lacked the functionality to instantly show the generated recipe to the user and provide a shopping list on the spot. This made it difficult for users to smoothly shop and reduce the effort required for cooking.

[0810] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0811] In this invention, the server includes a means for allowing a user to input information about the inventory of a home refrigerator, family composition, and cooking equipment owned by the user, a means for storing the input information in a database, and a means for acquiring sales information from nearby stores. This allows recipes to be displayed using a smart device in a physical store. Furthermore, by adding a means for generating a shopping list based on the recipe and guiding the user through shopping in the store, the user can shop efficiently and reduce the effort required for cooking.

[0812] "Means for allowing users to input information on the inventory of their household refrigerators, their family composition, and the cooking equipment they own" refers to a method in which users input the ingredients, number of people in their household, and the cooking equipment they can use, and collect that information.

[0813] The term "means for storing the input information in a database" refers to a system or device for electronically storing and managing information input by a user.

[0814] "Means for obtaining sale information from nearby stores" refers to a method or system for obtaining sale information at a store in cooperation with the store.

[0815] "Means for suggesting recipes generated based on the stored information and the acquired sale information" refers to a method or system for generating and suggesting appropriate recipes based on stored household information and sale information.

[0816] "Means for sending the proposed recipe to a user's terminal" refers to a method or system for sending the generated recipe to an electronic terminal, such as a smartphone or tablet.

[0817] "Means for displaying the recipe using a smart device in a physical store" refers to a method or system for displaying a recipe using a smart device while the user is in the physical store.

[0818] "Means for generating a shopping list based on the recipe and guiding shopping in the store" refers to a method or system for listing ingredients required for the generated recipe and allowing efficient shopping in the store based on that list.

[0819] The present invention is a system that collects household information from users and generates optimal recipes based on that information. This system operates in cooperation with three entities: a server, a terminal, and a user.

[0820] Entering and collecting user information

[0821] User

[0822] Users input information about their home refrigerator inventory, family composition, and cooking utensils via smart devices such as smartphones, tablets, smart glasses, etc. This information is temporarily stored on the user's device and then sent to the server.

[0823] Terminal

[0824] After receiving the information entered by the user, the device sends the data to the server, including the types and quantities of ingredients in the refrigerator, the number of family members and their food preferences, and the cooking equipment installed in the home.

[0825] server

[0826] The server stores the information sent from the device in a database that stores detailed household information for each user.

[0827] Collecting special sale information

[0828] The server periodically collects sales information by calling the sales information API in cooperation with nearby stores. The sales information of the stores is obtained from this API and saved in a database. This allows the latest sales information to be reflected in real time.

[0829] Recipe Generation

[0830] The server retrieves the user's household information and sales information from the database, and then uses generative AI (e.g., OpenAI's GPT-3 model) to generate a recipe based on this information. The generated recipe includes optimal cooking steps and a list of ingredients, taking into account the current refrigerator inventory and sales information.

[0831] Recipe presentation and selection

[0832] The server sends the generated recipe to the user's smart device terminal. It presents several recipe suggestions so that the user can choose from multiple recipes. The terminal displays the recipes sent from the server and offers options to the user. The user selects the recipe that best suits their preferences from the suggested recipes. The selected information is sent back to the server via the terminal.

[0833] Displaying recipes and generating shopping lists in physical stores

[0834] The user can view the recipes using a smart device in a physical store. The smart device displays the recipes based on the user's household information and sales information. The smart device also generates a shopping list based on the generated recipes, guiding the user to efficiently shop in the store.

[0835] For example, suppose User A has 6 eggs, 500ml of milk, 3 carrots, and 200g of chicken in his refrigerator. User A also inputs that he has a family of 3 people and owns an oven, microwave, and blender. The server uses a generation AI based on sale information obtained from nearby stores to generate the following recipe:

[0836] "Vegetable Omelette": Steps for making an omelet in a frying pan using eggs, carrots, and tomatoes.

[0837] "Chicken and Tomato Salad": Steps for making a salad of boiled chicken, tomatoes, and lettuce.

[0838] An example of a generated prompt is:

[0839] "The refrigerator contains {'eggs': 6, 'milk': '500ml', 'carrot': 3, 'chicken': '200g'}. There are three people in the family. The kitchen appliances are ['oven', 'microwave', 'blender']. The sale information is ['tomatoes: 100 yen / bag', 'beef: 500 yen / kg']. Please suggest a recipe based on this."

[0840] Users can view and select recipes using their smart devices in a physical store. Based on their selection, the system generates a shopping list with the ingredients they need, allowing users to shop efficiently in the store and reduce the effort required for cooking. In this way, the system of the present invention can maximize the use of each household's needs and resources to provide an efficient and satisfying dining experience.

[0841] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0842] Step 1:

[0843] A user uses a smart device (such as a smartphone, tablet, or smart glasses) to input information about the inventory of their home refrigerator, their family composition, and the cooking appliances they own. This input information is temporarily stored in the smart device's application.

[0844] Input: refrigerator inventory information, family composition, cooking utensil information

[0845] Output: Temporary data storage (smart device)

[0846] Step 2:

[0847] The device receives the information entered by the user and transmits the data to the server, including the types and quantities of ingredients in the refrigerator, the number of family members and their food preferences, and the cooking utensils available in the home.

[0848] Input: Home information entered by the user

[0849] Output: Send data to the server

[0850] Step 3:

[0851] The server stores the information sent from the devices in a database that stores detailed household information for each individual user.

[0852] Input: Home information sent from the device

[0853] Output: Save data to database

[0854] Step 4:

[0855] The server periodically collects sales information from nearby stores and calls the sales information API. This API retrieves sales information from the stores in real time and stores it in a database.

[0856] Input: Data retrieval request from the special sale information API

[0857] Output: Save the acquired sales information in a database

[0858] Step 5:

[0859] The server retrieves the user's household information and special sale information from the database, and then generates a recipe using a generative AI model (e.g., OpenAI's GPT-3). A prompt is entered into the generative AI model to generate an appropriate recipe. An example of a prompt is as follows:

[0860] "The refrigerator contains {'eggs': 6, 'milk': '500ml', 'carrot': 3, 'chicken': '200g'}. There are three people in the family. The kitchen appliances are ['oven', 'microwave', 'blender']. The sale information is ['tomatoes: 100 yen / bag', 'beef: 500 yen / kg']. Please suggest a recipe based on this."

[0861] Input: User's home information, special offer information, prompt text

[0862] Output: Generated recipe list

[0863] Step 6:

[0864] The server sends the generated recipe to the user's smart device, where several recipes are presented to the user so that the user can choose from them.

[0865] Input: Generated recipe list

[0866] Output: Send recipe to smart device

[0867] Step 7:

[0868] The user checks the recipes presented on the smart device and selects the recipe that suits their taste. The selected recipe information is then sent back to the server via the device.

[0869] Input: Multiple recipe suggestions, user selection

[0870] Output: Selected recipe information sent to server via terminal

[0871] Step 8:

[0872] The user displays the selected recipe using a smart device in a physical store, and the smart device generates a shopping list of ingredients based on the selected recipe, helping the user to shop efficiently in the store.

[0873] Input: User selected recipe

[0874] Output: Shopping list generation and in-store shopping guidance

[0875] The above steps enable optimal recipe suggestions based on household information and special sale information, as well as efficient shopping in physical stores.

[0876] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0877] This system generates optimal recipes by taking into account the user's household information and emotional state, and performs automatic cooking in cooperation with a cooking robot. This system functions through the cooperation of four entities: a server, a terminal, a user, and an emotion engine.

[0878] Entering and collecting user information

[0879] User

[0880] Users enter information about their refrigerator inventory, household composition, and cooking appliances through a dedicated application or web portal. This information is stored on the user's device and then sent to the server.

[0881] Terminal

[0882] The device receives information entered by the user, formats it, and sends it to a server. This data includes the types and quantities of food in the refrigerator, the number of family members and their food preferences, and the cooking equipment in the home.

[0883] server

[0884] The server stores the user information sent from the terminal in a database, which stores detailed household information for each user.

[0885] Collecting special sale information

[0886] server

[0887] The server periodically collects sales information by calling the sales information API in cooperation with nearby stores. The sales information of the stores is obtained from this API and saved in a database. This allows the latest sales information to be reflected in real time.

[0888] Recipe generation and emotional engine utilization

[0889] server

[0890] The server retrieves the user information and special sale information stored in the database and passes them to the generative AI engine, which then generates the optimal recipe. The generated recipe includes a list of ingredients and cooking instructions.

[0891] Emotion Engine

[0892] The emotion engine recognizes the user's emotional state in real time and transmits the emotion data to the server, which reflects the user's detailed emotional state, such as whether they are relaxed or stressed.

[0893] server

[0894] The server optimizes the generated recipes based on the emotion data sent from the emotion engine. For example, if the user is feeling stressed, it will prioritize recipes that are quick and easy to prepare.

[0895] Recipe presentation and selection

[0896] server

[0897] The server sends the optimized recipe to the user's device and presents several options so that the user can choose from multiple recipes.

[0898] Terminal

[0899] The terminal displays the recipe sent from the server and offers the user options.

[0900] User

[0901] The user selects the recipe that best suits their taste from the suggested recipes, and the selected information is sent back to the server via the terminal.

[0902] Automated cooking and delivery services

[0903] server

[0904] The server sends the recipe selected by the user to the cooking robot and instructs it to start cooking. The cooking robot automatically starts cooking based on the instructions sent.

[0905] cooking robot

[0906] The cooking robot automatically cooks according to the recipe information received from the server, and after cooking is complete, it sends a cooking completion notification to the server.

[0907] server

[0908] The server receives the cooking completion notification and issues delivery instructions to the delivery service as needed.

[0909] Community Collaboration

[0910] User

[0911] Users can share their leftover ingredients and cooking utensils with other users through the community platform, which helps reduce food waste and makes efficient use of the ingredients and cooking utensils they need.

[0912] server

[0913] The server manages transaction information between users and performs matching as appropriate. This information is saved in a database and used the next time a recipe is generated.

[0914] Specific examples

[0915] For example, suppose User B has six eggs, 500ml of milk, three carrots, and 200g of chicken in his refrigerator. User B also inputs that he has three family members, some of whom are vegetarian and some of whom prefer low-carb meals. Furthermore, suppose he owns an oven, a microwave, and a blender. If User B's emotional state indicates "stress," the server will optimize the recipe suggestions based on this emotional data.

[0916] The server has recorded in its database sales information obtained from nearby stores that tomatoes are 100 yen per bag and beef is 500 yen per kg. Based on this information, the server uses generative AI to generate the following recipe:

[0917] Vegetable Omelette: Steps to make an omelet in a frying pan using eggs, carrots, and tomatoes.

[0918] Chicken and Tomato Salad: Steps for a salad made with boiled chicken, tomatoes, and lettuce.

[0919] Furthermore, taking into account User B's stress level, the server prioritizes an "omelette with lots of vegetables," which can be cooked in a short time. User B selects this recipe, and the server sends instructions to the cooking robot. The cooking robot automatically cooks the omelette based on these instructions and serves it to User B at the set time, or delivers it via a delivery service.

[0920] In this way, the system of the present invention can provide an efficient and satisfying dining experience by taking into maximum consideration the needs and resources of each household, as well as the user's emotional state, thereby improving QoL.

[0921] The processing flow will be explained below.

[0922] Step 1:

[0923] A user accesses an application or web portal and enters information about the household refrigerator inventory, household composition, and cooking equipment they own, such as six eggs, 500ml of milk, three carrots, and 200g of chicken.

[0924] Step 2:

[0925] The terminal receives user-entered information, formats it, and sends it to the server. Specifically, the entered information is converted into JSON format data and sent to the server using the HTTPS protocol.

[0926] Step 3:

[0927] The server stores the user information received from the device in a database, including refrigerator inventory, family composition (e.g., three people in a family, vegetarian, low-carb preference), and a list of cooking appliances (oven, microwave, blender).

[0928] sql

[0929] INSERT INTO user_data (user_id, refrigerator, family, tools)

[0930] VALUES ("user123", '{"eggs":6,"milk":"500ml","carrots":3,"chicken":"200g"}', '{"members":3,"preferences":["vegetarian","low-carb"]}', '["oven","microwave","blender"]');

[0931] Step 4:

[0932] The server calls the special sale information API to get special sale information from nearby stores, such as tomatoes for 100 yen per bag and beef for 500 yen per kg.

[0933] json

[0934] {

[0935] "special_offers": {

[0936] "tomato": "100yen / bag",

[0937] "beef": "500yen / kg"

[0938] }

[0939] }

[0940] Step 5:

[0941] The server stores the acquired sale information in a database.

[0942] sql

[0943] INSERT INTO special_offers (store_id, item, price)

[0944] VALUES ("store456", "tomato", "100yen / bag"), ("store456", "beef", "500yen / kg");

[0945] Step 6:

[0946] The server retrieves the saved user information and sale information from the database and passes it to the generation AI engine.

[0947] sql

[0948] SELECT FROM user_data WHERE user_id = "user123";

[0949] SELECT FROM special_offers WHERE store_id = "store456";

[0950] Step 7:

[0951] The generative AI engine generates optimal recipes based on the acquired data. The generated recipes include a list of ingredients and cooking instructions. For example, it generates recipes for a "vegetable omelet" and a "chicken and tomato salad."

[0952] json

[0953] {

[0954] "recipes": [

[0955] {

[0956] "name": "Vegetable Omelette",

[0957] "ingredients": ["eggs", "carrots", "tomato"],

[0958] "instructions": "Mix eggs, chop vegetables, and fry in a frying pan. Finally, pour in the eggs to make an omelet."

[0959] },

[0960] {

[0961] "name": "Chicken and Tomato Salad",

[0962] "ingredients": ["chicken", "tomato"],

[0963] "instructions": "Boil the chicken and serve it as a salad with tomatoes."

[0964] }

[0965] ]

[0966] }

[0967] Step 8:

[0968] The emotion engine recognizes the user's emotional state in real time and sends the emotion data to the server. For example, if the user is feeling stressed, the information is sent to the server.

[0969] Step 9:

[0970] The server optimizes the generated recipes based on the emotional data from the emotion engine. For example, if the user is feeling stressed, it will prioritize suggesting an "omelette with lots of vegetables," which can be cooked quickly and easily.

[0971] Step 10:

[0972] The server sends a list of optimized recipes to the user's device.

[0973] Step 11:

[0974] The device displays the recipes and offers options to the user, who can then select the recipe that best suits their taste.

[0975] Step 12:

[0976] The recipe information selected by the user is again transmitted to the server via the terminal.

[0977] Step 13:

[0978] The server sends the selected recipe to the cooking robot and instructs it to start cooking.

[0979] Step 14:

[0980] The cooking robot automatically starts cooking according to the recipe information received from the server.

[0981] Step 15:

[0982] After the cooking robot completes the cooking, it sends a cooking completion notification to the server.

[0983] Step 16:

[0984] The server receives the cooking completion notification and issues delivery instructions to the delivery service as needed.

[0985] Step 17:

[0986] A delivery service delivers pre-cooked meals to a user's location.

[0987] Step 18:

[0988] Users share leftover ingredients and cooking utensils with other users through a community platform.

[0989] Step 19:

[0990] The server stores the information posted on the community platform in a database and uses it the next time a recipe is generated.

[0991] This process dynamically and efficiently generates optimal recipes that take into account the user's household information, sales information, and emotional state, and works in conjunction with a cooking robot to achieve hassle-free cooking. Furthermore, the community platform also promotes efficient resource utilization.

[0992] Example 2

[0993] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0994] In modern households, managing refrigerator inventory and planning meals requires time and effort, while food waste is becoming a serious problem. Furthermore, recipe suggestions that take into account the user's emotional state may not be adequately tailored to the user's needs and resources. Therefore, there is a need for a system that can provide efficient and optimal meal plans that take into account food inventory information, sales information, and the user's emotional state.

[0995] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0996] In this invention, the server includes means for having a user input information on household food inventory, household configuration, and owned cooking utensils, means for organizing the input information and transmitting the input information to the server, means for saving the input information in a database, means for acquiring sales information from nearby stores, means for generating recipes using a generative AI model based on the saved information and the acquired sales information, means for recognizing the user's emotional state in real time and transmitting the data to the server, means for optimizing the generated recipe based on the emotional data, means for transmitting the optimized recipe to the user's terminal, and means for the user to select a recipe. This enables optimal recipe suggestions and automatic cooking based on the user's household information and emotional state, thereby reducing food waste and improving user satisfaction.

[0997] "Food inventory information" is data regarding the type, quantity, and condition of food stored in a household refrigerator or pantry.

[0998] "Family composition" is information about the number of family members in the user's household and their individual food preferences.

[0999] "Cooking utensils" is information about the types and number of utensils and equipment used for cooking that the user owns.

[1000] "Special Offers" is data about discounts and specially priced food items offered at nearby stores.

[1001] A "generative AI model" is an artificial intelligence algorithm that generates optimal recipes based on the user's household information and special sale information.

[1002] "Emotional state" is data that assesses a user's current emotions and stress levels in real time.

[1003] "Optimization" is the process of improving the generated recipe by taking into account the user's emotional state and other variables.

[1004] A "server" is a computer system that receives, stores, analyzes input information, and performs recipe generation and optimization.

[1005] A "terminal" is a device that a user uses to enter information, receive recipes, and make selections.

[1006] "Automatic cooking" is the process in which a cooking robot automatically prepares a dish based on a recipe instructed by the server.

[1007] A "community platform" is an online or digital system for users to share leftover ingredients and cooking equipment.

[1008] This system generates optimal recipes by taking into account the user's household information and emotional state, and performs automatic cooking in cooperation with a cooking robot. This system functions in cooperation with multiple entities: a server, a terminal, a user, and an emotion engine.

[1009] Entering and collecting user information

[1010] Users use a dedicated application or web portal to enter information about the food inventory in their refrigerator, their family composition, and the cooking equipment they own. This information is stored on the user's device and then sent to a server. The device then organizes the information entered by the user and sends it to the server. This data includes the types and quantities of ingredients in the refrigerator, the number of family members and their food preferences, and the cooking equipment installed in the home.

[1011] The server stores the user information sent from the device in a database. Detailed household information for each user is stored in the database. The server periodically collects sales information by linking with nearby stores and calling the sales information API. The sales information for each store obtained from this API is stored in the database. This allows the latest sales information to be reflected in real time.

[1012] Recipe generation and emotional engine utilization

[1013] The server retrieves saved user information and special sale information from the database and provides a prompt to the AI ​​engine based on this information. The AI ​​engine then generates the optimal recipe. For example, the prompt is "Please generate a recipe for a family of three that is vegetarian-friendly." The generated recipe includes a list of ingredients and cooking instructions.

[1014] The emotion engine recognizes the user's emotional state in real time and sends that emotional data to the server. This data reflects the user's detailed emotional state, such as whether they are relaxed or stressed. The server optimizes the generated recipes based on the emotional data sent from the emotion engine. For example, if the user is feeling stressed, it will prioritize recipes that are quick and easy to prepare.

[1015] Recipe presentation and selection

[1016] The server sends the optimized recipe to the user's device. The user is presented with several options to choose from. The device displays the recipe sent from the server and provides the user with options.

[1017] The user selects the recipe that best suits their taste from the suggested recipes, and this selection information is sent back to the server via the terminal.

[1018] Automated cooking and delivery services

[1019] The server sends the recipe selected by the user to the cooking robot and instructs it to start cooking. The cooking robot automatically starts cooking based on the instructions sent. After cooking is complete, it sends a cooking completion notification to the server.

[1020] The server receives the cooking completion notification and issues delivery instructions to the delivery service as needed.

[1021] Community Collaboration

[1022] Through the community platform, users can share their surplus ingredients and cooking utensils with other users. This reduces food waste and allows for efficient use of necessary ingredients and cooking utensils. The server manages transaction information between users and performs matching. This information is stored in a database and will be used the next time recipes are generated.

[1023] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1024] Step 1:

[1025] Users use a dedicated application or web portal to enter information about their household food inventory, household composition, and cooking equipment they own. For example, they enter information such as the number of eggs, 500ml of milk, 3 carrots, and 200g of chicken in the refrigerator. The data entered by the user is stored on the device.

[1026] Input: household food inventory information, household composition, cooking utensil information

[1027] Output: User information stored on the device

[1028] Step 2:

[1029] The device receives the information entered by the user, formats it, and sends it to the server. The data is converted to, for example, JSON format and sent to the server. Specific operations include the device organizing the user data, packetizing it in the appropriate format, and sending it to the server via the network.

[1030] Input: User information stored on the device

[1031] Output: User information sent to the server

[1032] Step 3:

[1033] The server receives the user information sent from the device and stores it in a database. Specifically, it parses the received JSON data and stores it in a structured database. This allows detailed household information for each user to be accumulated in the database.

[1034] Input: User information sent to the server

[1035] Output: User information stored in the database

[1036] Step 4:

[1037] The server periodically connects with nearby stores and calls the sale information API to collect sale information. From this API, sale information such as "Tomatoes for 100 yen / bag, beef for 500 yen / kg" is obtained. The obtained information is stored in a database.

[1038] Input: Special sale information obtained from the special sale information API

[1039] Output: Special sale information stored in the database

[1040] Step 5:

[1041] The server retrieves user information and special sale information from the database and provides a prompt to the AI ​​engine. For example, the prompt might be, "Please generate a recipe that is suitable for a family of three and suitable for vegetarians." The AI ​​engine then generates the optimal recipe based on this prompt. The generated recipe includes a list of ingredients and cooking instructions.

[1042] Input: User information and special offer information retrieved from the database, prompt text

[1043] Output: Recipe generated by the generative AI engine

[1044] Step 6:

[1045] The emotion engine recognizes the user's emotional state in real time and sends that data to the server. For example, it determines whether the user is feeling stressed and provides that information to the server. The server then optimizes the generated recipes based on the data sent from the emotion engine. Specifically, it suggests recipes that are quick and easy to prepare to users who are feeling stressed.

[1046] Input: Real-time user emotional state data

[1047] Output: Optimized recipe taking into account emotional state

[1048] Step 7:

[1049] The server generates an optimized recipe and sends it to the user's device. The device displays the received recipe and offers the user several options. For example, it presents recipes such as "vegetable omelet" and "chicken and tomato salad."

[1050] Input: Optimized recipe generated on the server

[1051] Output: Recipe choices displayed on terminal

[1052] Step 8:

[1053] The user uses the terminal to select a desired recipe from the multiple recipes presented, and this selection information is sent back to the server from the terminal.

[1054] Input: User selected recipe

[1055] Output: Selected recipe information sent to the server

[1056] Step 9:

[1057] The server sends the selected recipe information to the cooking robot and instructs it to start cooking. The cooking robot automatically cooks the food based on the instructed recipe. For example, after cooking a "vegetable omelet," it sends a cooking completion notification to the server.

[1058] Input: Selected recipe information

[1059] Output: Automatic cooking by cooking robot, notification of cooking completion

[1060] Step 10:

[1061] The server receives a cooking completion notification from the cooking robot and, based on this, issues delivery instructions to the delivery service as needed. If the user wants to receive the food at home, the server delivers it directly.

[1062] Input: Cooking completion notification

[1063] Output: Delivery instructions to the delivery service, delivery of food to the user

[1064] The above processing steps enable optimal recipe suggestions and automatic cooking that take into account the user's home information and emotional state.

[1065] (Application example 2)

[1066] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1067] In today's busy lifestyles, preparing meals at home is a time-consuming and labor-intensive task that can be stressful for users. It is particularly difficult to select efficient, cost-effective recipes while taking into account the household's ingredient inventory and the family's food preferences. Furthermore, the user's emotional state also influences meal selection, so there is a need for recipe suggestions that reduce stress and promote relaxation. The present invention solves these problems by providing a system that proposes optimal recipes that take the user's emotional state into account and enables automatic cooking.

[1068] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1069] In this invention, the server includes: means for prompting a user to input information about the user's refrigerator inventory, family composition, and cooking utensils; means for saving the input information in a database; means for acquiring sales information from nearby stores; means for suggesting recipes generated using a generative AI model based on the saved information and the acquired sales information; means for acquiring and recognizing the user's emotional state; means for optimizing the generated recipes based on the acquired emotional state; and means for transmitting the suggested recipes to the user's device and providing multiple options. This allows for the creation of recipes that reflect the user's refrigerator inventory and family food preferences, and furthermore, by taking the user's emotional state into consideration, more optimized recipe suggestions are possible. Furthermore, providing multiple options allows for flexible responses to the user's lifestyle and mood at any given time, thereby reducing the effort and stress of cooking.

[1070] A "user" is an entity that uses this system to input information about the inventory of a household refrigerator, family composition, and cooking equipment, and receives optimal recipe suggestions.

[1071] "Home refrigerator inventory information" is information about the types and quantities of food ingredients that a user keeps in their home.

[1072] "Family composition" is information about the members of the user's household, including the number of people living in the household, their age groups, food preferences, and allergy information.

[1073] "Information about cooking utensils owned" is information about the types and quantities of cooking utensils that the user owns in the home.

[1074] The "database" is an electronic information storage device for storing and managing information input by the user and acquired sale information.

[1075] "Special sale information" is information about discounts and special sales on food and ingredients obtained from nearby stores.

[1076] A "generative AI model" is an artificial intelligence algorithm that automatically generates optimal recipes based on user information and special sale information.

[1077] "User's emotional state" is information about the mental state of the user, such as whether the user is relaxed or stressed.

[1078] "Optimizing a recipe" refers to processing the generated recipe according to the user's emotional state to make more appropriate cooking suggestions.

[1079] A "user terminal" is an electronic device, such as a smartphone or tablet, through which a user inputs information and receives suggested recipes.

[1080] A "cooking robot" is a mechanical device that automatically cooks food based on a recipe selected by the user.

[1081] A "community platform" is a shared space on the Internet where users can share and exchange leftover ingredients and cooking utensils.

[1082] This system generates optimal recipes by taking into account the user's household information and emotional state, and performs automatic cooking in cooperation with food delivery and cooking robots. This system functions through the cooperation of four entities: a server, a terminal, a user, and an emotion engine.

[1083] Entering and collecting user information

[1084] User

[1085] Users enter information about their refrigerator inventory, household composition, and cooking appliances through a dedicated application or web portal. This information is stored on the user's device and then sent to the server.

[1086] Terminal

[1087] The device receives information entered by the user, formats it, and sends it to a server. This data includes the types and quantities of food in the refrigerator, the number of family members and their food preferences, and the cooking equipment in the home.

[1088] Collecting special sale information

[1089] server

[1090] The server periodically collects sales information by calling the sales information API in cooperation with nearby stores. The sales information of the stores is obtained from this API and saved in a database. This allows the latest sales information to be reflected in real time.

[1091] Recipe generation and emotional engine utilization

[1092] server

[1093] The server retrieves the user information and special sale information stored in the database and passes them to the generative AI engine, which then generates the optimal recipe. The generated recipe includes a list of ingredients and cooking instructions.

[1094] Emotion Engine

[1095] The emotion engine recognizes the user's emotional state in real time and transmits the emotion data to the server, which reflects the user's detailed emotional state, such as whether they are relaxed or stressed.

[1096] server

[1097] The server optimizes the generated recipes based on the emotion data sent from the emotion engine. For example, if the user is feeling stressed, it will prioritize recipes that are quick and easy to prepare.

[1098] Recipe presentation and selection

[1099] server

[1100] The server sends the optimized recipe to the user's device and presents several options so that the user can choose from multiple recipes.

[1101] Terminal

[1102] The terminal displays the recipe sent from the server and offers the user options.

[1103] User

[1104] The user selects the recipe that best suits their taste from the suggested recipes, and the selected information is sent back to the server via the terminal.

[1105] Automated cooking and delivery services

[1106] server

[1107] The server sends the recipe selected by the user to the cooking robot and instructs it to start cooking. The cooking robot automatically starts cooking based on the instructions sent.

[1108] cooking robot

[1109] The cooking robot automatically cooks according to the recipe information received from the server, and after cooking is complete, it sends a cooking completion notification to the server.

[1110] server

[1111] The server receives the cooking completion notification and issues delivery instructions to the delivery service as needed.

[1112] Community Collaboration

[1113] User

[1114] Users can share their leftover ingredients and cooking utensils with other users through the community platform, which helps reduce food waste and makes efficient use of the ingredients and cooking utensils they need.

[1115] server

[1116] The server manages transaction information between users and performs matching as appropriate. This information is saved in a database and used the next time a recipe is generated.

[1117] Specific examples

[1118] For example, suppose User B has six eggs, 500ml of milk, three carrots, and 200g of chicken in his refrigerator. User B also inputs that he has three family members, some of whom are vegetarian and some of whom prefer low-carb meals. Furthermore, suppose he owns an oven, a microwave, and a blender. If User B's emotional state indicates "stress," the server will optimize the recipe suggestions based on this emotional data.

[1119] The server has recorded in its database sales information obtained from nearby stores that tomatoes are 100 yen per bag and beef is 500 yen per kg. Based on this information, the server uses generative AI to generate the following recipe:

[1120] Vegetable Omelette: Steps to make an omelet in a frying pan using eggs, carrots, and tomatoes.

[1121] Chicken and Tomato Salad: Steps for a salad made with boiled chicken, tomatoes, and lettuce.

[1122] Furthermore, taking into account User B's stress level, the system prioritizes the suggestion of an "omelette with lots of vegetables," which can be cooked in a short time. User B selects this recipe, and the server sends instructions to the cooking robot. The cooking robot automatically cooks the omelette based on these instructions and serves it to User B at the set time, or delivers it via a delivery service. In this way, the system of the present invention provides an efficient and satisfying dining experience, taking into full consideration the needs and resources of each household, as well as the user's emotional state, and thereby improving QoL.

[1123] Example prompt for a generative AI model:

[1124] Generate recipes using the ingredients found in a home refrigerator (6 eggs, 500ml of milk, 3 carrots, 200g of chicken) and cooking equipment (oven, microwave, blender) taking into account the emotional state of stress. Also, reflect sales information (e.g., tomatoes for 100 yen / bag, beef for 500 yen / kg).

[1125] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1126] Step 1:

[1127] Enter and submit user information

[1128] Users enter information about their refrigerator inventory, family composition, and cooking equipment through a dedicated application or web portal. The input data includes specific food information, such as "6 eggs, 500ml of milk, 3 carrots, and 200g of chicken," as well as the number of family members and their food preferences. This information is sent to the device, where it is formatted and then sent to the server.

[1129] Input: Home refrigerator inventory information, family composition, cooking equipment information

[1130] Output: Household information data sent to the server

[1131] Step 2:

[1132] Collecting special sale information

[1133] The server periodically calls the sales information API from nearby stores to collect sales information. Through this API, sales information such as "Tomatoes for 100 yen / bag, beef for 500 yen / kg" is obtained and saved in a database.

[1134] Input: Special sale information obtained from the special sale information API

[1135] Output: Special sale information data stored in a database

[1136] Step 3:

[1137] Obtaining the user's emotional state

[1138] The device uses an emotion engine to recognize the user's emotional state in real time and determines the user's mental state. For example, if the device determines that the user is feeling "stressed," that information is sent to the server.

[1139] Input: User emotion data obtained by the emotion engine

[1140] Output: Emotion data sent to the server

[1141] Step 4:

[1142] Recipe Generation

[1143] The server uses a generative AI model to generate the optimal recipe based on user information and special sale information obtained from the database. It passes a prompt to the generative AI model, generating a prompt such as the following:

[1144] Generate recipes using the ingredients found in a home refrigerator (6 eggs, 500ml of milk, 3 carrots, 200g of chicken) and cooking equipment (oven, microwave, blender) taking into account the emotional state of stress. Also, reflect sales information (e.g., tomatoes for 100 yen / bag, beef for 500 yen / kg).

[1145] Input: User information, sale information, emotion data

[1146] Output: The generated optimal recipe

[1147] Step 5:

[1148] Recipe optimization

[1149] The server optimizes the generated recipes based on the acquired emotional state. For example, if the user is feeling stressed, the server will prioritize the easy-to-make "vegetable omelet" from the multiple generated recipes.

[1150] Input: Generated recipe, emotional state data

[1151] Output: Optimized recipe

[1152] Step 6:

[1153] Recipe presentation and selection

[1154] The server sends the optimized recipe to the terminal and presents multiple options to the user. The user selects the recipe that best suits their preferences and sends the selection back to the server via the terminal.

[1155] Input: Optimized recipe

[1156] Output: The recipe selected by the user

[1157] Step 7:

[1158] Starting automatic cooking

[1159] The server sends the recipe selected by the user to the cooking robot and instructs it to start cooking. The cooking robot automatically starts cooking based on the instructions sent. After cooking is complete, it sends a cooking completion notification to the server. If necessary, it issues delivery instructions to a delivery service.

[1160] Input: User selected recipe

[1161] Output: Food cooked and served

[1162] Step 8:

[1163] Community Collaboration

[1164] The server manages a community platform for users to share leftover ingredients and cooking utensils among themselves. Through the platform, users can share their leftover ingredients and cooking utensils with other users and efficiently use the ingredients and utensils they need.

[1165] Input: Information about leftover ingredients and cooking utensils

[1166] Output: Matching information of shared or exchanged ingredients and cooking utensils

[1167] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1168] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1169] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1170] [Third embodiment]

[1171] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1172] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1173] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1174] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1175] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1176] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1177] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1178] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1179] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1180] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1181] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1182] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1183] This invention is a system that collects household information from users and generates optimal recipes based on that information. This system operates in cooperation with three entities: a server, a terminal, and the user.

[1184] Entering and collecting user information

[1185] User

[1186] Users enter information about their refrigerator inventory, household composition, and cooking appliances through an application or web portal. This information is stored on the user's device and then sent to the server.

[1187] Terminal

[1188] After receiving the information entered by the user, the device sends the data to the server, including the types and quantities of ingredients in the refrigerator, the number of family members and their food preferences, and the cooking equipment installed in the home.

[1189] server

[1190] The server stores the information sent from the device in a database that stores detailed household information for each user.

[1191] Collecting special sale information

[1192] server

[1193] The server periodically collects sales information by calling the sales information API in cooperation with nearby stores. The sales information of the stores is obtained from this API and saved in a database. This allows the latest sales information to be reflected in real time.

[1194] Recipe Generation

[1195] server

[1196] The server retrieves the user's household information and sales information from the database, and uses generative AI to generate recipes based on this information. The generated recipes include optimal cooking steps and ingredient lists that take into account the current refrigerator inventory and sales information.

[1197] Recipe presentation and selection

[1198] server

[1199] The server sends the generated recipe to the user's device and presents several recipe suggestions so that the user can choose from multiple recipes.

[1200] Terminal

[1201] The terminal displays the recipe sent from the server and offers the user options.

[1202] User

[1203] The user selects the recipe that best suits their taste from the suggested recipes, and the selected information is sent back to the server via the terminal.

[1204] Automated cooking and delivery services

[1205] server

[1206] The server sends the recipe selected by the user to the cooking robot and provides cooking instructions, and the cooking robot automatically starts cooking based on the instructions sent.

[1207] cooking robot

[1208] The cooking robot automatically cooks according to the recipe received from the server and serves the finished dish to the user, or works with a delivery service to deliver it to the user.

[1209] Community Collaboration

[1210] User

[1211] Users can share their leftover ingredients and cooking utensils with other users through the community platform, which helps reduce food waste and makes efficient use of the ingredients and cooking utensils they need.

[1212] server

[1213] The server manages transaction information between users and performs appropriate matching. This information is stored in a database and is reflected in the next recipe suggestions.

[1214] Specific examples

[1215] For example, suppose User A has 6 eggs, 500ml of milk, 3 carrots, and 200g of chicken in his refrigerator. User A also inputs that he has a family of 3 people, some of whom are vegetarian and some of whom have low-carb diets. He also owns an oven, a microwave, and a blender.

[1216] The server records in its database the sale information it has obtained from nearby stores, such as tomatoes at 100 yen per bag and beef at 500 yen per kg.

[1217] Based on this information, the server uses a generative AI to generate the following recipe:

[1218] Vegetable Omelette: Steps to make an omelet in a frying pan using eggs, carrots, and tomatoes.

[1219] Chicken and Tomato Salad: Steps for a salad made with boiled chicken, tomatoes, and lettuce.

[1220] The user selects a vegetable omelet from the list, and the server sends this information to the cooking robot, which then automatically cooks the omelet and serves it to the user at the scheduled time or delivers it via a delivery service.

[1221] In this way, the system of the present invention can make the most of each household's needs and resources to provide an efficient and satisfying dining experience.

[1222] The processing flow will be explained below.

[1223] Step 1:

[1224] A user accesses an application or web portal and enters information about the inventory of their home refrigerator, their family members, and the cooking appliances they own.

[1225] Step 2:

[1226] The terminal receives user input information, formats it, and sends it to the server.

[1227] Step 3:

[1228] The server stores the user information received from the device in a database, which includes refrigerator inventory information, family composition, and a list of cooking utensils.

[1229] Step 4:

[1230] The server calls the sale information API to get sale information from nearby stores. The sale information includes the store name, sale items, and prices.

[1231] Step 5:

[1232] The server stores the acquired sale information in a database.

[1233] Step 6:

[1234] The server retrieves the saved user information and sale information from the database and passes it to the generation AI engine.

[1235] Step 7:

[1236] The generative AI engine generates optimal recipes based on the acquired data, including a list of ingredients and cooking instructions.

[1237] Step 8:

[1238] The server sends the list of generated recipes to the user's terminal.

[1239] Step 9:

[1240] The user checks the recipes sent and selects the recipe they like.

[1241] Step 10:

[1242] The terminal again transmits the user's selection information to the server.

[1243] Step 11:

[1244] The server sends the selected recipe to the cooking robot and instructs it to start cooking.

[1245] Step 12:

[1246] The cooking robot automatically starts cooking according to the recipe information received from the server.

[1247] Step 13:

[1248] After the cooking robot completes the cooking, it sends a cooking completion notification to the server.

[1249] Step 14:

[1250] The server receives the cooking completion notification and issues delivery instructions to the delivery service as needed.

[1251] Step 15:

[1252] A delivery service delivers pre-cooked meals to a user's location.

[1253] Step 16:

[1254] Users share leftover ingredients and cooking utensils with other users through a community platform.

[1255] Step 17:

[1256] The server stores the information posted on the community platform in a database and uses it the next time a recipe is generated.

[1257] This process dynamically generates optimal recipes based on the user's household information and sales information, and in cooperation with the cooking robot, achieves hassle-free cooking. Furthermore, the community platform also promotes the effective use of resources.

[1258] Example 1

[1259] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1260] Traditionally, when cooking at home, users had to come up with their own recipes based on the ingredients and cooking utensils they had, which was time-consuming and labor-intensive. Furthermore, there was a lack of mechanisms for efficient shopping using sales information or for sharing ingredients and cooking utensils with other users. To solve these issues, a system that provides an efficient and satisfying dining experience is needed.

[1261] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1262] In this invention, the server includes means for allowing users to input information about their home refrigerator inventory, family composition, and cooking utensils; means for saving the input information in a database; means for acquiring sales information from nearby stores; means for generating and suggesting recipes using a generative AI model; and means for transmitting the suggested recipes to the user's device. This allows users to easily obtain optimal recipes based on the ingredients, cooking utensils, and sales information available at home, enabling them to cook efficiently and save time and effort. Furthermore, sharing ingredients and cooking utensils among users promotes the reduction of food waste and the efficient use of resources.

[1263] A "user" is an entity that uses the system to provide refrigerator inventory information, family composition, and cooking utensil information.

[1264] A "database" is a storage device for storing and managing information entered by users, sale information, and the like.

[1265] "Special sale information" is information obtained from nearby stores about products being sold at discounted prices.

[1266] A "generative AI model" is an artificial intelligence algorithm that generates recipes based on accumulated data.

[1267] "Suggestion" is the act of showing a recipe generated by a generative AI model to a user.

[1268] A "terminal" is an electronic device through which a user inputs information and receives suggested recipes.

[1269] A "cooking device" is a machine that automatically cooks food based on a recipe sent from a server.

[1270] A "community platform" is an online gathering place where users can share leftover ingredients and cooking utensils.

[1271] This invention is a system that collects household information from users and generates optimal recipes based on that information. This system operates in cooperation with three entities: a server, a terminal, and the user.

[1272] Entering and collecting user information

[1273] User

[1274] Users enter information about their refrigerator inventory, household composition, and cooking appliances through an application or web portal. This information is stored on the user's device and then sent to the server.

[1275] Terminal

[1276] After receiving the information entered by the user, the device sends the data to the server, including the types and quantities of ingredients in the refrigerator, the number of family members and their food preferences, and the cooking equipment installed in the home.

[1277] server

[1278] The server stores the information sent from the device in a database that stores detailed household information for each user.

[1279] Collecting special sale information

[1280] server

[1281] The server periodically collects sales information by calling the sales information API in cooperation with nearby stores. The sales information of the stores is obtained from this API and saved in a database. This allows the latest sales information to be reflected in real time.

[1282] Recipe Generation

[1283] server

[1284] The server retrieves the user's household information and sales information from the database, and then uses the generative AI model to generate recipes based on this information. The generated recipes include optimal cooking steps and ingredient lists that take into account the current refrigerator inventory and sales information.

[1285] Recipe presentation and selection

[1286] server

[1287] The server sends the generated recipe to the user's device and presents several recipe suggestions so that the user can choose from multiple recipes.

[1288] Terminal

[1289] The terminal displays the recipe sent from the server and offers the user options.

[1290] User

[1291] The user selects the recipe that best suits their taste from the suggested recipes, and the selected information is sent back to the server via the terminal.

[1292] Automated cooking and delivery services

[1293] server

[1294] The server sends the recipe selected by the user to the cooking device and provides cooking instructions, and the cooking device automatically starts cooking based on the transmitted instructions.

[1295] cooking equipment

[1296] The cooking device automatically cooks the food according to the recipe received from the server and serves the finished dish to the user, or delivers it to the user in cooperation with a delivery service.

[1297] Community Collaboration

[1298] User

[1299] Users can share their leftover ingredients and cooking utensils with other users through the community platform, which helps reduce food waste and makes efficient use of the ingredients and cooking utensils they need.

[1300] server

[1301] The server manages transaction information between users and performs appropriate matching. This information is stored in a database and is reflected in the next recipe suggestions.

[1302] Specific examples

[1303] For example, suppose User A has 6 eggs, 500ml of milk, 3 carrots, and 200g of chicken in his refrigerator. User A also inputs that he has a family of 3 people, some of whom are vegetarian and some of whom have low-carb diets. He also owns an oven, a microwave, and a blender.

[1304] The server records in its database the sale information it has obtained from nearby stores, such as tomatoes at 100 yen per bag and beef at 500 yen per kg.

[1305] With this information, the server uses a generative AI model to generate the following recipe:

[1306] Vegetable Omelet: How to make an omelet in a frying pan using eggs, carrots, and tomatoes

[1307] Chicken and Tomato Salad: A salad made with boiled chicken, tomatoes, and lettuce.

[1308] The user selects the vegetable omelet, and the server sends this information to the cooking device, which then automatically cooks the omelet and serves it to the user at the scheduled time or delivers it via a delivery service.

[1309] In this way, the system of the present invention can make the most of each household's needs and resources to provide an efficient and satisfying dining experience.

[1310] Prompt Sentence Examples

[1311] "In the refrigerator, there are six eggs, 500ml of milk, three carrots, and 200g of chicken. There are three people in my family, and some are vegetarian and some are low-carb. I also have an oven, microwave, and blender. I'm currently seeing special offers on tomatoes for 100 yen per bag and beef for 500 yen per kg. Please generate the best recipe for me."

[1312] By inputting this prompt into a generative AI model, the optimal recipe based on the user's needs can be generated.

[1313] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1314] Step 1:

[1315] Entering user information

[1316] Users open an application or web portal and enter information about their refrigerator inventory, household composition, and cooking appliances, which is temporarily stored on the user's device.

[1317] Input: Ingredients in the refrigerator (e.g., 6 eggs, 500 ml of milk, 3 carrots, 200 g of chicken), family size (e.g., 3 people), cooking equipment (e.g., oven, microwave, blender)

[1318] Output: Temporarily stores the household information entered by the user.

[1319] Step 2:

[1320] Sending user information

[1321] The terminal transmits the information input by the user to the server. Specifically, the information is transmitted when the send button is clicked.

[1322] Input: Home information entered by the user

[1323] Output: Home information received by the server

[1324] Step 3:

[1325] Saving user information

[1326] The server stores the home information sent from the device in a database, and the stored data is organized by user.

[1327] Input: Home information sent from the device

[1328] Output: Household information stored in a database

[1329] Step 4:

[1330] Collecting special sale information

[1331] The server periodically calls the sales information API to obtain the latest sales information from nearby stores, and stores this information in a database.

[1332] Input: Call the sale information API

[1333] Output: Special sale information stored in the database

[1334] Step 5:

[1335] Recipe Generation

[1336] The server retrieves the user's household information and special sale information from the database and inputs it as a prompt into the generative AI model, which then generates the optimal recipe based on that information.

[1337] Input: Household information and special sale information obtained from the database

[1338] Output: Recipes generated by the generative AI model

[1339] Specific operation: The server generates a prompt (e.g., "In the refrigerator, there are 6 eggs, 500ml of milk, 3 carrots, and 200g of chicken. There are 3 people in the family, and some prefer vegetarian and low-carb foods. There is also an oven, microwave, and blender. There are special offers on tomatoes for 100 yen per bag and beef for 500 yen per kg. Please generate the best recipe.") and sends it to the generative AI model.

[1340] Step 6:

[1341] Submitting a Recipe

[1342] The server transmits the generated recipe to the user's terminal.

[1343] Input: Recipes generated by a generative AI model

[1344] Output: Recipe sent to terminal

[1345] Step 7:

[1346] Viewing Recipes

[1347] The terminal displays the recipe sent from the server to the user.

[1348] Input: Recipe sent from the server

[1349] Output: The recipe that is displayed to the user

[1350] Specific operation: Recipes for "Vegetable Omelette" and "Chicken and Tomato Salad" displayed on the application screen

[1351] Step 8:

[1352] Recipe Selection

[1353] The user selects a recipe from the displayed list, and the selection information is sent to the server via the terminal.

[1354] Input: User selected recipe

[1355] Output: Selected recipe information sent to the server

[1356] Step 9:

[1357] Cooking Instructions

[1358] The server transmits the recipe selected by the user to the cooking device and issues cooking instructions.

[1359] Input: Selected recipe information

[1360] Output: Cooking instructions sent to the cooking device

[1361] Step 10:

[1362] Automatic cooking

[1363] The cooking device automatically starts cooking based on the recipe information received from the server, and once the cooking is complete, it serves the food to the user or delivers it via a delivery service.

[1364] Input: Cooking instructions from the server

[1365] Output: Finished dish

[1366] Specific operation: The cooking device automatically cooks an omelet and serves the finished product to the user.

[1367] Step 11:

[1368] Community Collaboration

[1369] Users share their leftover ingredients and cooking utensils with other users through the community platform, and this sharing information is sent to the server.

[1370] Input: Information on leftover ingredients and cooking utensils shared by users

[1371] Output: Shared information sent to the server

[1372] Step 12:

[1373] Management of transaction information

[1374] The server manages transaction information between users and performs matching as appropriate. Matching information is stored in a database and reflected in the next recipe suggestions.

[1375] Input: Shared ingredients and cooking utensil information

[1376] Output: Transaction information stored in the database

[1377] Specific operation: Matching transaction information between users such as "I have two eggs left" or "I need two eggs"

[1378] Through these steps, the system generates and provides optimal recipes tailored to the needs of each household, providing an efficient and satisfying dining experience.

[1379] (Application example 1)

[1380] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1381] Conventional recipe suggestion systems were able to generate recipes based on information about the household refrigerator's inventory, household configuration, and cooking utensils, but they lacked support for users to have an efficient and effective shopping experience in a physical store. They also lacked the functionality to instantly show the generated recipe to the user and provide a shopping list on the spot. This made it difficult for users to smoothly shop and reduce the effort required for cooking.

[1382] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1383] In this invention, the server includes a means for allowing a user to input information about the inventory of a home refrigerator, family composition, and cooking equipment owned by the user, a means for storing the input information in a database, and a means for acquiring sales information from nearby stores. This allows recipes to be displayed using a smart device in a physical store. Furthermore, by adding a means for generating a shopping list based on the recipe and guiding the user through shopping in the store, the user can shop efficiently and reduce the effort required for cooking.

[1384] "Means for allowing users to input information on the inventory of their household refrigerators, their family composition, and the cooking equipment they own" refers to a method in which users input the ingredients, number of people in their household, and the cooking equipment they can use, and collect that information.

[1385] The term "means for storing the input information in a database" refers to a system or device for electronically storing and managing information input by a user.

[1386] "Means for obtaining sale information from nearby stores" refers to a method or system for obtaining sale information at a store in cooperation with the store.

[1387] "Means for suggesting recipes generated based on the stored information and the acquired sale information" refers to a method or system for generating and suggesting appropriate recipes based on stored household information and sale information.

[1388] "Means for sending the proposed recipe to a user's terminal" refers to a method or system for sending the generated recipe to an electronic terminal, such as a smartphone or tablet.

[1389] "Means for displaying the recipe using a smart device in a physical store" refers to a method or system for displaying a recipe using a smart device while the user is in the physical store.

[1390] "Means for generating a shopping list based on the recipe and guiding shopping in the store" refers to a method or system for listing ingredients required for the generated recipe and allowing efficient shopping in the store based on that list.

[1391] The present invention is a system that collects household information from users and generates optimal recipes based on that information. This system operates in cooperation with three entities: a server, a terminal, and a user.

[1392] Entering and collecting user information

[1393] User

[1394] Users input information about their home refrigerator inventory, family composition, and cooking utensils via smart devices such as smartphones, tablets, smart glasses, etc. This information is temporarily stored on the user's device and then sent to the server.

[1395] Terminal

[1396] After receiving the information entered by the user, the device sends the data to the server, including the types and quantities of ingredients in the refrigerator, the number of family members and their food preferences, and the cooking equipment installed in the home.

[1397] server

[1398] The server stores the information sent from the device in a database that stores detailed household information for each user.

[1399] Collecting special sale information

[1400] The server periodically collects sales information by calling the sales information API in cooperation with nearby stores. The sales information of the stores is obtained from this API and saved in a database. This allows the latest sales information to be reflected in real time.

[1401] Recipe Generation

[1402] The server retrieves the user's household information and sales information from the database, and then uses generative AI (e.g., OpenAI's GPT-3 model) to generate a recipe based on this information. The generated recipe includes optimal cooking steps and a list of ingredients, taking into account the current refrigerator inventory and sales information.

[1403] Recipe presentation and selection

[1404] The server sends the generated recipe to the user's smart device terminal. It presents several recipe suggestions so that the user can choose from multiple recipes. The terminal displays the recipes sent from the server and offers options to the user. The user selects the recipe that best suits their preferences from the suggested recipes. The selected information is sent back to the server via the terminal.

[1405] Displaying recipes and generating shopping lists in physical stores

[1406] The user can view the recipes using a smart device in a physical store. The smart device displays the recipes based on the user's household information and sales information. The smart device also generates a shopping list based on the generated recipes, guiding the user to efficiently shop in the store.

[1407] For example, suppose User A has 6 eggs, 500ml of milk, 3 carrots, and 200g of chicken in his refrigerator. User A also inputs that he has a family of 3 people and owns an oven, microwave, and blender. The server uses a generation AI based on sale information obtained from nearby stores to generate the following recipe:

[1408] "Vegetable Omelette": Steps for making an omelet in a frying pan using eggs, carrots, and tomatoes.

[1409] "Chicken and Tomato Salad": Steps for making a salad of boiled chicken, tomatoes, and lettuce.

[1410] An example of a generated prompt is:

[1411] "The refrigerator contains {'eggs': 6, 'milk': '500ml', 'carrot': 3, 'chicken': '200g'}. There are three people in the family. The kitchen appliances are ['oven', 'microwave', 'blender']. The sale information is ['tomatoes: 100 yen / bag', 'beef: 500 yen / kg']. Please suggest a recipe based on this."

[1412] Users can view and select recipes using their smart devices in a physical store. Based on their selection, the system generates a shopping list with the ingredients they need, allowing users to shop efficiently in the store and reduce the effort required for cooking. In this way, the system of the present invention can maximize the use of each household's needs and resources to provide an efficient and satisfying dining experience.

[1413] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1414] Step 1:

[1415] A user uses a smart device (such as a smartphone, tablet, or smart glasses) to input information about the inventory of their home refrigerator, their family composition, and the cooking appliances they own. This input information is temporarily stored in the smart device's application.

[1416] Input: refrigerator inventory information, family composition, cooking utensil information

[1417] Output: Temporary data storage (smart device)

[1418] Step 2:

[1419] The device receives the information entered by the user and transmits the data to the server, including the types and quantities of ingredients in the refrigerator, the number of family members and their food preferences, and the cooking utensils available in the home.

[1420] Input: Home information entered by the user

[1421] Output: Send data to the server

[1422] Step 3:

[1423] The server stores the information sent from the devices in a database that stores detailed household information for each individual user.

[1424] Input: Home information sent from the device

[1425] Output: Save data to database

[1426] Step 4:

[1427] The server periodically collects sales information from nearby stores and calls the sales information API. This API retrieves sales information from the stores in real time and stores it in a database.

[1428] Input: Data retrieval request from the special sale information API

[1429] Output: Save the acquired sales information in a database

[1430] Step 5:

[1431] The server retrieves the user's household information and special sale information from the database, and then generates a recipe using a generative AI model (e.g., OpenAI's GPT-3). A prompt is entered into the generative AI model to generate an appropriate recipe. An example of a prompt is as follows:

[1432] "The refrigerator contains {'eggs': 6, 'milk': '500ml', 'carrot': 3, 'chicken': '200g'}. There are three people in the family. The kitchen appliances are ['oven', 'microwave', 'blender']. The sale information is ['tomatoes: 100 yen / bag', 'beef: 500 yen / kg']. Please suggest a recipe based on this."

[1433] Input: User's home information, special offer information, prompt text

[1434] Output: Generated recipe list

[1435] Step 6:

[1436] The server sends the generated recipe to the user's smart device, where several recipes are presented to the user so that the user can choose from them.

[1437] Input: Generated recipe list

[1438] Output: Send recipe to smart device

[1439] Step 7:

[1440] The user checks the recipes presented on the smart device and selects the recipe that suits their taste. The selected recipe information is then sent back to the server via the device.

[1441] Input: Multiple recipe suggestions, user selection

[1442] Output: Selected recipe information sent to server via terminal

[1443] Step 8:

[1444] The user displays the selected recipe using a smart device in a physical store, and the smart device generates a shopping list of ingredients based on the selected recipe, helping the user to shop efficiently in the store.

[1445] Input: User selected recipe

[1446] Output: Shopping list generation and in-store shopping guidance

[1447] The above steps enable optimal recipe suggestions based on household information and special sale information, as well as efficient shopping in physical stores.

[1448] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1449] This system generates optimal recipes by taking into account the user's household information and emotional state, and performs automatic cooking in cooperation with a cooking robot. This system functions through the cooperation of four entities: a server, a terminal, a user, and an emotion engine.

[1450] Entering and collecting user information

[1451] User

[1452] Users enter information about their refrigerator inventory, household composition, and cooking appliances through a dedicated application or web portal. This information is stored on the user's device and then sent to the server.

[1453] Terminal

[1454] The device receives information entered by the user, formats it, and sends it to a server. This data includes the types and quantities of food in the refrigerator, the number of family members and their food preferences, and the cooking equipment in the home.

[1455] server

[1456] The server stores the user information sent from the terminal in a database, which stores detailed household information for each user.

[1457] Collecting special sale information

[1458] server

[1459] The server periodically collects sales information by calling the sales information API in cooperation with nearby stores. The sales information of the stores is obtained from this API and saved in a database. This allows the latest sales information to be reflected in real time.

[1460] Recipe generation and emotional engine utilization

[1461] server

[1462] The server retrieves the user information and special sale information stored in the database and passes them to the generative AI engine, which then generates the optimal recipe. The generated recipe includes a list of ingredients and cooking instructions.

[1463] Emotion Engine

[1464] The emotion engine recognizes the user's emotional state in real time and transmits the emotion data to the server, which reflects the user's detailed emotional state, such as whether they are relaxed or stressed.

[1465] server

[1466] The server optimizes the generated recipes based on the emotion data sent from the emotion engine. For example, if the user is feeling stressed, it will prioritize recipes that are quick and easy to prepare.

[1467] Recipe presentation and selection

[1468] server

[1469] The server sends the optimized recipe to the user's device and presents several options so that the user can choose from multiple recipes.

[1470] Terminal

[1471] The terminal displays the recipe sent from the server and offers the user options.

[1472] User

[1473] The user selects the recipe that best suits their taste from the suggested recipes, and the selected information is sent back to the server via the terminal.

[1474] Automated cooking and delivery services

[1475] server

[1476] The server sends the recipe selected by the user to the cooking robot and instructs it to start cooking. The cooking robot automatically starts cooking based on the instructions sent.

[1477] cooking robot

[1478] The cooking robot automatically cooks according to the recipe information received from the server, and after cooking is complete, it sends a cooking completion notification to the server.

[1479] server

[1480] The server receives the cooking completion notification and issues delivery instructions to the delivery service as needed.

[1481] Community Collaboration

[1482] User

[1483] Users can share their leftover ingredients and cooking utensils with other users through the community platform, which helps reduce food waste and makes efficient use of the ingredients and cooking utensils they need.

[1484] server

[1485] The server manages transaction information between users and performs matching as appropriate. This information is saved in a database and used the next time a recipe is generated.

[1486] Specific examples

[1487] For example, suppose User B has six eggs, 500ml of milk, three carrots, and 200g of chicken in his refrigerator. User B also inputs that he has three family members, some of whom are vegetarian and some of whom prefer low-carb meals. Furthermore, suppose he owns an oven, a microwave, and a blender. If User B's emotional state indicates "stress," the server will optimize the recipe suggestions based on this emotional data.

[1488] The server has recorded in its database sales information obtained from nearby stores that tomatoes are 100 yen per bag and beef is 500 yen per kg. Based on this information, the server uses generative AI to generate the following recipe:

[1489] Vegetable Omelette: Steps to make an omelet in a frying pan using eggs, carrots, and tomatoes.

[1490] Chicken and Tomato Salad: Steps for a salad made with boiled chicken, tomatoes, and lettuce.

[1491] Furthermore, taking into account User B's stress level, the server prioritizes an "omelette with lots of vegetables," which can be cooked in a short time. User B selects this recipe, and the server sends instructions to the cooking robot. The cooking robot automatically cooks the omelette based on these instructions and serves it to User B at the set time, or delivers it via a delivery service.

[1492] In this way, the system of the present invention can provide an efficient and satisfying dining experience by taking into maximum consideration the needs and resources of each household, as well as the user's emotional state, thereby improving QoL.

[1493] The processing flow will be explained below.

[1494] Step 1:

[1495] A user accesses an application or web portal and enters information about the household refrigerator inventory, household composition, and cooking equipment they own, such as six eggs, 500ml of milk, three carrots, and 200g of chicken.

[1496] Step 2:

[1497] The terminal receives user-entered information, formats it, and sends it to the server. Specifically, the entered information is converted into JSON format data and sent to the server using the HTTPS protocol.

[1498] Step 3:

[1499] The server stores the user information received from the device in a database, including refrigerator inventory, family composition (e.g., three people in a family, vegetarian, low-carb preference), and a list of cooking appliances (oven, microwave, blender).

[1500] sql

[1501] INSERT INTO user_data (user_id, refrigerator, family, tools)

[1502] VALUES ("user123", '{"eggs":6,"milk":"500ml","carrots":3,"chicken":"200g"}', '{"members":3,"preferences":["vegetarian","low-carb"]}', '["oven","microwave","blender"]');

[1503] Step 4:

[1504] The server calls the special sale information API to get special sale information from nearby stores, such as tomatoes for 100 yen per bag and beef for 500 yen per kg.

[1505] json

[1506] {

[1507] "special_offers": {

[1508] "tomato": "100yen / bag",

[1509] "beef": "500yen / kg"

[1510] }

[1511] }

[1512] Step 5:

[1513] The server stores the acquired sale information in a database.

[1514] sql

[1515] INSERT INTO special_offers (store_id, item, price)

[1516] VALUES ("store456", "tomato", "100yen / bag"), ("store456", "beef", "500yen / kg");

[1517] Step 6:

[1518] The server retrieves the saved user information and sale information from the database and passes it to the generation AI engine.

[1519] sql

[1520] SELECT FROM user_data WHERE user_id = "user123";

[1521] SELECT FROM special_offers WHERE store_id = "store456";

[1522] Step 7:

[1523] The generative AI engine generates optimal recipes based on the acquired data. The generated recipes include a list of ingredients and cooking instructions. For example, it generates recipes for a "vegetable omelet" and a "chicken and tomato salad."

[1524] json

[1525] {

[1526] "recipes": [

[1527] {

[1528] "name": "Vegetable Omelette",

[1529] "ingredients": ["eggs", "carrots", "tomato"],

[1530] "instructions": "Mix eggs, chop vegetables, and fry in a frying pan. Finally, pour in the eggs to make an omelet."

[1531] },

[1532] {

[1533] "name": "Chicken and Tomato Salad",

[1534] "ingredients": ["chicken", "tomato"],

[1535] "instructions": "Boil the chicken and serve it as a salad with tomatoes."

[1536] }

[1537] ]

[1538] }

[1539] Step 8:

[1540] The emotion engine recognizes the user's emotional state in real time and sends the emotion data to the server. For example, if the user is feeling stressed, the information is sent to the server.

[1541] Step 9:

[1542] The server optimizes the generated recipes based on the emotional data from the emotion engine. For example, if the user is feeling stressed, it will prioritize suggesting an "omelette with lots of vegetables," which can be cooked quickly and easily.

[1543] Step 10:

[1544] The server sends a list of optimized recipes to the user's device.

[1545] Step 11:

[1546] The device displays the recipes and offers options to the user, who can then select the recipe that best suits their taste.

[1547] Step 12:

[1548] The recipe information selected by the user is again transmitted to the server via the terminal.

[1549] Step 13:

[1550] The server sends the selected recipe to the cooking robot and instructs it to start cooking.

[1551] Step 14:

[1552] The cooking robot automatically starts cooking according to the recipe information received from the server.

[1553] Step 15:

[1554] After the cooking robot completes the cooking, it sends a cooking completion notification to the server.

[1555] Step 16:

[1556] The server receives the cooking completion notification and issues delivery instructions to the delivery service as needed.

[1557] Step 17:

[1558] A delivery service delivers pre-cooked meals to a user's location.

[1559] Step 18:

[1560] Users share leftover ingredients and cooking utensils with other users through a community platform.

[1561] Step 19:

[1562] The server stores the information posted on the community platform in a database and uses it the next time a recipe is generated.

[1563] This process dynamically and efficiently generates optimal recipes that take into account the user's household information, sales information, and emotional state, and works in conjunction with a cooking robot to achieve hassle-free cooking. Furthermore, the community platform also promotes efficient resource utilization.

[1564] Example 2

[1565] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1566] In modern households, managing refrigerator inventory and planning meals requires time and effort, while food waste is becoming a serious problem. Furthermore, recipe suggestions that take into account the user's emotional state may not be adequately tailored to the user's needs and resources. Therefore, there is a need for a system that can provide efficient and optimal meal plans that take into account food inventory information, sales information, and the user's emotional state.

[1567] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1568] In this invention, the server includes means for having a user input information on household food inventory, household configuration, and owned cooking utensils, means for organizing the input information and transmitting the input information to the server, means for saving the input information in a database, means for acquiring sales information from nearby stores, means for generating recipes using a generative AI model based on the saved information and the acquired sales information, means for recognizing the user's emotional state in real time and transmitting the data to the server, means for optimizing the generated recipe based on the emotional data, means for transmitting the optimized recipe to the user's terminal, and means for the user to select a recipe. This enables optimal recipe suggestions and automatic cooking based on the user's household information and emotional state, thereby reducing food waste and improving user satisfaction.

[1569] "Food inventory information" is data regarding the type, quantity, and condition of food stored in a household refrigerator or pantry.

[1570] "Family composition" is information about the number of family members in the user's household and their individual food preferences.

[1571] "Cooking utensils" is information about the types and number of utensils and equipment used for cooking that the user owns.

[1572] "Special Offers" is data about discounts and specially priced food items offered at nearby stores.

[1573] A "generative AI model" is an artificial intelligence algorithm that generates optimal recipes based on the user's household information and special sale information.

[1574] "Emotional state" is data that assesses a user's current emotions and stress levels in real time.

[1575] "Optimization" is the process of improving the generated recipe by taking into account the user's emotional state and other variables.

[1576] A "server" is a computer system that receives, stores, analyzes input information, and performs recipe generation and optimization.

[1577] A "terminal" is a device that a user uses to enter information, receive recipes, and make selections.

[1578] "Automatic cooking" is the process in which a cooking robot automatically prepares a dish based on a recipe instructed by the server.

[1579] A "community platform" is an online or digital system for users to share leftover ingredients and cooking equipment.

[1580] This system generates optimal recipes by taking into account the user's household information and emotional state, and performs automatic cooking in cooperation with a cooking robot. This system functions in cooperation with multiple entities: a server, a terminal, a user, and an emotion engine.

[1581] Entering and collecting user information

[1582] Users use a dedicated application or web portal to enter information about the food inventory in their refrigerator, their family composition, and the cooking equipment they own. This information is stored on the user's device and then sent to a server. The device then organizes the information entered by the user and sends it to the server. This data includes the types and quantities of ingredients in the refrigerator, the number of family members and their food preferences, and the cooking equipment installed in the home.

[1583] The server stores the user information sent from the device in a database. Detailed household information for each user is stored in the database. The server periodically collects sales information by linking with nearby stores and calling the sales information API. The sales information for each store obtained from this API is stored in the database. This allows the latest sales information to be reflected in real time.

[1584] Recipe generation and emotional engine utilization

[1585] The server retrieves saved user information and special sale information from the database and provides a prompt to the AI ​​engine based on this information. The AI ​​engine then generates the optimal recipe. For example, the prompt is "Please generate a recipe for a family of three that is vegetarian-friendly." The generated recipe includes a list of ingredients and cooking instructions.

[1586] The emotion engine recognizes the user's emotional state in real time and sends that emotional data to the server. This data reflects the user's detailed emotional state, such as whether they are relaxed or stressed. The server optimizes the generated recipes based on the emotional data sent from the emotion engine. For example, if the user is feeling stressed, it will prioritize recipes that are quick and easy to prepare.

[1587] Recipe presentation and selection

[1588] The server sends the optimized recipe to the user's device. The user is presented with several options to choose from. The device displays the recipe sent from the server and provides the user with options.

[1589] The user selects the recipe that best suits their taste from the suggested recipes, and this selection information is sent back to the server via the terminal.

[1590] Automated cooking and delivery services

[1591] The server sends the recipe selected by the user to the cooking robot and instructs it to start cooking. The cooking robot automatically starts cooking based on the instructions sent. After cooking is complete, it sends a cooking completion notification to the server.

[1592] The server receives the cooking completion notification and issues delivery instructions to the delivery service as needed.

[1593] Community Collaboration

[1594] Through the community platform, users can share their surplus ingredients and cooking utensils with other users. This reduces food waste and allows for efficient use of necessary ingredients and cooking utensils. The server manages transaction information between users and performs matching. This information is stored in a database and will be used the next time recipes are generated.

[1595] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1596] Step 1:

[1597] Users use a dedicated application or web portal to enter information about their household food inventory, household composition, and cooking equipment they own. For example, they enter information such as the number of eggs, 500ml of milk, 3 carrots, and 200g of chicken in the refrigerator. The data entered by the user is stored on the device.

[1598] Input: household food inventory information, household composition, cooking utensil information

[1599] Output: User information stored on the device

[1600] Step 2:

[1601] The device receives the information entered by the user, formats it, and sends it to the server. The data is converted to, for example, JSON format and sent to the server. Specific operations include the device organizing the user data, packetizing it in the appropriate format, and sending it to the server via the network.

[1602] Input: User information stored on the device

[1603] Output: User information sent to the server

[1604] Step 3:

[1605] The server receives the user information sent from the device and stores it in a database. Specifically, it parses the received JSON data and stores it in a structured database. This allows detailed household information for each user to be accumulated in the database.

[1606] Input: User information sent to the server

[1607] Output: User information stored in the database

[1608] Step 4:

[1609] The server periodically connects with nearby stores and calls the sale information API to collect sale information. From this API, sale information such as "Tomatoes for 100 yen / bag, beef for 500 yen / kg" is obtained. The obtained information is stored in a database.

[1610] Input: Special sale information obtained from the special sale information API

[1611] Output: Special sale information stored in the database

[1612] Step 5:

[1613] The server retrieves user information and special sale information from the database and provides a prompt to the AI ​​engine. For example, the prompt might be, "Please generate a recipe that is suitable for a family of three and suitable for vegetarians." The AI ​​engine then generates the optimal recipe based on this prompt. The generated recipe includes a list of ingredients and cooking instructions.

[1614] Input: User information and special offer information retrieved from the database, prompt text

[1615] Output: Recipe generated by the generative AI engine

[1616] Step 6:

[1617] The emotion engine recognizes the user's emotional state in real time and sends that data to the server. For example, it determines whether the user is feeling stressed and provides that information to the server. The server then optimizes the generated recipes based on the data sent from the emotion engine. Specifically, it suggests recipes that are quick and easy to prepare to users who are feeling stressed.

[1618] Input: Real-time user emotional state data

[1619] Output: Optimized recipe taking into account emotional state

[1620] Step 7:

[1621] The server generates an optimized recipe and sends it to the user's device. The device displays the received recipe and offers the user several options. For example, it presents recipes such as "vegetable omelet" and "chicken and tomato salad."

[1622] Input: Optimized recipe generated on the server

[1623] Output: Recipe choices displayed on terminal

[1624] Step 8:

[1625] The user uses the terminal to select a desired recipe from the multiple recipes presented, and this selection information is sent back to the server from the terminal.

[1626] Input: User selected recipe

[1627] Output: Selected recipe information sent to the server

[1628] Step 9:

[1629] The server sends the selected recipe information to the cooking robot and instructs it to start cooking. The cooking robot automatically cooks the food based on the instructed recipe. For example, after cooking a "vegetable omelet," it sends a cooking completion notification to the server.

[1630] Input: Selected recipe information

[1631] Output: Automatic cooking by cooking robot, notification of cooking completion

[1632] Step 10:

[1633] The server receives a cooking completion notification from the cooking robot and, based on this, issues delivery instructions to the delivery service as needed. If the user wants to receive the food at home, the server delivers it directly.

[1634] Input: Cooking completion notification

[1635] Output: Delivery instructions to the delivery service, delivery of food to the user

[1636] The above processing steps enable optimal recipe suggestions and automatic cooking that take into account the user's home information and emotional state.

[1637] (Application example 2)

[1638] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1639] In today's busy lifestyles, preparing meals at home is a time-consuming and labor-intensive task that can be stressful for users. It is particularly difficult to select efficient, cost-effective recipes while taking into account the household's ingredient inventory and the family's food preferences. Furthermore, the user's emotional state also influences meal selection, so there is a need for recipe suggestions that reduce stress and promote relaxation. The present invention solves these problems by providing a system that proposes optimal recipes that take the user's emotional state into account and enables automatic cooking.

[1640] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1641] In this invention, the server includes: means for prompting a user to input information about the user's refrigerator inventory, family composition, and cooking utensils; means for saving the input information in a database; means for acquiring sales information from nearby stores; means for suggesting recipes generated using a generative AI model based on the saved information and the acquired sales information; means for acquiring and recognizing the user's emotional state; means for optimizing the generated recipes based on the acquired emotional state; and means for transmitting the suggested recipes to the user's device and providing multiple options. This allows for the creation of recipes that reflect the user's refrigerator inventory and family food preferences, and furthermore, by taking the user's emotional state into consideration, more optimized recipe suggestions are possible. Furthermore, providing multiple options allows for flexible responses to the user's lifestyle and mood at any given time, thereby reducing the effort and stress of cooking.

[1642] A "user" is an entity that uses this system to input information about the inventory of a household refrigerator, family composition, and cooking equipment, and receives optimal recipe suggestions.

[1643] "Home refrigerator inventory information" is information about the types and quantities of food ingredients that a user keeps in their home.

[1644] "Family composition" is information about the members of the user's household, including the number of people living in the household, their age groups, food preferences, and allergy information.

[1645] "Information about cooking utensils owned" is information about the types and quantities of cooking utensils that the user owns in the home.

[1646] The "database" is an electronic information storage device for storing and managing information input by the user and acquired sale information.

[1647] "Special sale information" is information about discounts and special sales on food and ingredients obtained from nearby stores.

[1648] A "generative AI model" is an artificial intelligence algorithm that automatically generates optimal recipes based on user information and special sale information.

[1649] "User's emotional state" is information about the mental state of the user, such as whether the user is relaxed or stressed.

[1650] "Optimizing a recipe" refers to processing the generated recipe according to the user's emotional state to make more appropriate cooking suggestions.

[1651] A "user terminal" is an electronic device, such as a smartphone or tablet, through which a user inputs information and receives suggested recipes.

[1652] A "cooking robot" is a mechanical device that automatically cooks food based on a recipe selected by the user.

[1653] A "community platform" is a shared space on the Internet where users can share and exchange leftover ingredients and cooking utensils.

[1654] This system generates optimal recipes by taking into account the user's household information and emotional state, and performs automatic cooking in cooperation with food delivery and cooking robots. This system functions through the cooperation of four entities: a server, a terminal, a user, and an emotion engine.

[1655] Entering and collecting user information

[1656] User

[1657] Users enter information about their refrigerator inventory, household composition, and cooking appliances through a dedicated application or web portal. This information is stored on the user's device and then sent to the server.

[1658] Terminal

[1659] The device receives information entered by the user, formats it, and sends it to a server. This data includes the types and quantities of food in the refrigerator, the number of family members and their food preferences, and the cooking equipment in the home.

[1660] Collecting special sale information

[1661] server

[1662] The server periodically collects sales information by calling the sales information API in cooperation with nearby stores. The sales information of the stores is obtained from this API and saved in a database. This allows the latest sales information to be reflected in real time.

[1663] Recipe generation and emotional engine utilization

[1664] server

[1665] The server retrieves the user information and special sale information stored in the database and passes them to the generative AI engine, which then generates the optimal recipe. The generated recipe includes a list of ingredients and cooking instructions.

[1666] Emotion Engine

[1667] The emotion engine recognizes the user's emotional state in real time and transmits the emotion data to the server, which reflects the user's detailed emotional state, such as whether they are relaxed or stressed.

[1668] server

[1669] The server optimizes the generated recipes based on the emotion data sent from the emotion engine. For example, if the user is feeling stressed, it will prioritize recipes that are quick and easy to prepare.

[1670] Recipe presentation and selection

[1671] server

[1672] The server sends the optimized recipe to the user's device and presents several options so that the user can choose from multiple recipes.

[1673] Terminal

[1674] The terminal displays the recipe sent from the server and offers the user options.

[1675] User

[1676] The user selects the recipe that best suits their taste from the suggested recipes, and the selected information is sent back to the server via the terminal.

[1677] Automated cooking and delivery services

[1678] server

[1679] The server sends the recipe selected by the user to the cooking robot and instructs it to start cooking. The cooking robot automatically starts cooking based on the instructions sent.

[1680] cooking robot

[1681] The cooking robot automatically cooks according to the recipe information received from the server, and after cooking is complete, it sends a cooking completion notification to the server.

[1682] server

[1683] The server receives the cooking completion notification and issues delivery instructions to the delivery service as needed.

[1684] Community Collaboration

[1685] User

[1686] Users can share their leftover ingredients and cooking utensils with other users through the community platform, which helps reduce food waste and makes efficient use of the ingredients and cooking utensils they need.

[1687] server

[1688] The server manages transaction information between users and performs matching as appropriate. This information is saved in a database and used the next time a recipe is generated.

[1689] Specific examples

[1690] For example, suppose User B has six eggs, 500ml of milk, three carrots, and 200g of chicken in his refrigerator. User B also inputs that he has three family members, some of whom are vegetarian and some of whom prefer low-carb meals. Furthermore, suppose he owns an oven, a microwave, and a blender. If User B's emotional state indicates "stress," the server will optimize the recipe suggestions based on this emotional data.

[1691] The server has recorded in its database sales information obtained from nearby stores that tomatoes are 100 yen per bag and beef is 500 yen per kg. Based on this information, the server uses generative AI to generate the following recipe:

[1692] Vegetable Omelette: Steps to make an omelet in a frying pan using eggs, carrots, and tomatoes.

[1693] Chicken and Tomato Salad: Steps for a salad made with boiled chicken, tomatoes, and lettuce.

[1694] Furthermore, taking into account User B's stress level, the system prioritizes the suggestion of an "omelette with lots of vegetables," which can be cooked in a short time. User B selects this recipe, and the server sends instructions to the cooking robot. The cooking robot automatically cooks the omelette based on these instructions and serves it to User B at the set time, or delivers it via a delivery service. In this way, the system of the present invention provides an efficient and satisfying dining experience, taking into full consideration the needs and resources of each household, as well as the user's emotional state, and thereby improving QoL.

[1695] Example prompt for a generative AI model:

[1696] Generate recipes using the ingredients found in a home refrigerator (6 eggs, 500ml of milk, 3 carrots, 200g of chicken) and cooking equipment (oven, microwave, blender) taking into account the emotional state of stress. Also, reflect sales information (e.g., tomatoes for 100 yen / bag, beef for 500 yen / kg).

[1697] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1698] Step 1:

[1699] Enter and submit user information

[1700] Users enter information about their refrigerator inventory, family composition, and cooking equipment through a dedicated application or web portal. The input data includes specific food information, such as "6 eggs, 500ml of milk, 3 carrots, and 200g of chicken," as well as the number of family members and their food preferences. This information is sent to the device, where it is formatted and then sent to the server.

[1701] Input: Home refrigerator inventory information, family composition, cooking equipment information

[1702] Output: Household information data sent to the server

[1703] Step 2:

[1704] Collecting special sale information

[1705] The server periodically calls the sales information API from nearby stores to collect sales information. Through this API, sales information such as "Tomatoes for 100 yen / bag, beef for 500 yen / kg" is obtained and saved in a database.

[1706] Input: Special sale information obtained from the special sale information API

[1707] Output: Special sale information data stored in a database

[1708] Step 3:

[1709] Obtaining the user's emotional state

[1710] The device uses an emotion engine to recognize the user's emotional state in real time and determines the user's mental state. For example, if the device determines that the user is feeling "stressed," that information is sent to the server.

[1711] Input: User emotion data obtained by the emotion engine

[1712] Output: Emotion data sent to the server

[1713] Step 4:

[1714] Recipe Generation

[1715] The server uses a generative AI model to generate the optimal recipe based on user information and special sale information obtained from the database. It passes a prompt to the generative AI model, generating a prompt such as the following:

[1716] Generate recipes using the ingredients found in a home refrigerator (6 eggs, 500ml of milk, 3 carrots, 200g of chicken) and cooking equipment (oven, microwave, blender) taking into account the emotional state of stress. Also, reflect sales information (e.g., tomatoes for 100 yen / bag, beef for 500 yen / kg).

[1717] Input: User information, sale information, emotion data

[1718] Output: The generated optimal recipe

[1719] Step 5:

[1720] Recipe optimization

[1721] The server optimizes the generated recipes based on the acquired emotional state. For example, if the user is feeling stressed, the server will prioritize the easy-to-make "vegetable omelet" from the multiple generated recipes.

[1722] Input: Generated recipe, emotional state data

[1723] Output: Optimized recipe

[1724] Step 6:

[1725] Recipe presentation and selection

[1726] The server sends the optimized recipe to the terminal and presents multiple options to the user. The user selects the recipe that best suits their preferences and sends the selection back to the server via the terminal.

[1727] Input: Optimized recipe

[1728] Output: The recipe selected by the user

[1729] Step 7:

[1730] Starting automatic cooking

[1731] The server sends the recipe selected by the user to the cooking robot and instructs it to start cooking. The cooking robot automatically starts cooking based on the instructions sent. After cooking is complete, it sends a cooking completion notification to the server. If necessary, it issues delivery instructions to a delivery service.

[1732] Input: User selected recipe

[1733] Output: Food cooked and served

[1734] Step 8:

[1735] Community Collaboration

[1736] The server manages a community platform for users to share leftover ingredients and cooking utensils among themselves. Through the platform, users can share their leftover ingredients and cooking utensils with other users and efficiently use the ingredients and utensils they need.

[1737] Input: Information about leftover ingredients and cooking utensils

[1738] Output: Matching information of shared or exchanged ingredients and cooking utensils

[1739] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1740] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1741] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1742] [Fourth embodiment]

[1743] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1744] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1745] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1746] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1747] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1748] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1749] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1750] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1751] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1752] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1753] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1754] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1755] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1756] This invention is a system that collects household information from users and generates optimal recipes based on that information. This system operates in cooperation with three entities: a server, a terminal, and the user.

[1757] Entering and collecting user information

[1758] User

[1759] Users enter information about their refrigerator inventory, household composition, and cooking appliances through an application or web portal. This information is stored on the user's device and then sent to the server.

[1760] Terminal

[1761] After receiving the information entered by the user, the device sends the data to the server, including the types and quantities of ingredients in the refrigerator, the number of family members and their food preferences, and the cooking equipment installed in the home.

[1762] server

[1763] The server stores the information sent from the device in a database that stores detailed household information for each user.

[1764] Collecting special sale information

[1765] server

[1766] The server periodically collects sales information by calling the sales information API in cooperation with nearby stores. The sales information of the stores is obtained from this API and saved in a database. This allows the latest sales information to be reflected in real time.

[1767] Recipe Generation

[1768] server

[1769] The server retrieves the user's household information and sales information from the database, and uses generative AI to generate recipes based on this information. The generated recipes include optimal cooking steps and ingredient lists that take into account the current refrigerator inventory and sales information.

[1770] Recipe presentation and selection

[1771] server

[1772] The server sends the generated recipe to the user's device and presents several recipe suggestions so that the user can choose from multiple recipes.

[1773] Terminal

[1774] The terminal displays the recipe sent from the server and offers the user options.

[1775] User

[1776] The user selects the recipe that best suits their taste from the suggested recipes, and the selected information is sent back to the server via the terminal.

[1777] Automated cooking and delivery services

[1778] server

[1779] The server sends the recipe selected by the user to the cooking robot and provides cooking instructions, and the cooking robot automatically starts cooking based on the instructions sent.

[1780] cooking robot

[1781] The cooking robot automatically cooks according to the recipe received from the server and serves the finished dish to the user, or works with a delivery service to deliver it to the user.

[1782] Community Collaboration

[1783] User

[1784] Users can share their leftover ingredients and cooking utensils with other users through the community platform, which helps reduce food waste and makes efficient use of the ingredients and cooking utensils they need.

[1785] server

[1786] The server manages transaction information between users and performs appropriate matching. This information is stored in a database and is reflected in the next recipe suggestions.

[1787] Specific examples

[1788] For example, suppose User A has 6 eggs, 500ml of milk, 3 carrots, and 200g of chicken in his refrigerator. User A also inputs that he has a family of 3 people, some of whom are vegetarian and some of whom have low-carb diets. He also owns an oven, a microwave, and a blender.

[1789] The server records in its database the sale information it has obtained from nearby stores, such as tomatoes at 100 yen per bag and beef at 500 yen per kg.

[1790] Based on this information, the server uses a generative AI to generate the following recipe:

[1791] Vegetable Omelette: Steps to make an omelet in a frying pan using eggs, carrots, and tomatoes.

[1792] Chicken and Tomato Salad: Steps for a salad made with boiled chicken, tomatoes, and lettuce.

[1793] The user selects a vegetable omelet from the list, and the server sends this information to the cooking robot, which then automatically cooks the omelet and serves it to the user at the scheduled time or delivers it via a delivery service.

[1794] In this way, the system of the present invention can make the most of each household's needs and resources to provide an efficient and satisfying dining experience.

[1795] The processing flow will be explained below.

[1796] Step 1:

[1797] A user accesses an application or web portal and enters information about the inventory of their home refrigerator, their family members, and the cooking appliances they own.

[1798] Step 2:

[1799] The terminal receives user input information, formats it, and sends it to the server.

[1800] Step 3:

[1801] The server stores the user information received from the device in a database, which includes refrigerator inventory information, family composition, and a list of cooking utensils.

[1802] Step 4:

[1803] The server calls the sale information API to get sale information from nearby stores. The sale information includes the store name, sale items, and prices.

[1804] Step 5:

[1805] The server stores the acquired sale information in a database.

[1806] Step 6:

[1807] The server retrieves the saved user information and sale information from the database and passes it to the generation AI engine.

[1808] Step 7:

[1809] The generative AI engine generates optimal recipes based on the acquired data, including a list of ingredients and cooking instructions.

[1810] Step 8:

[1811] The server sends the list of generated recipes to the user's terminal.

[1812] Step 9:

[1813] The user checks the recipes sent and selects the recipe they like.

[1814] Step 10:

[1815] The terminal again transmits the user's selection information to the server.

[1816] Step 11:

[1817] The server sends the selected recipe to the cooking robot and instructs it to start cooking.

[1818] Step 12:

[1819] The cooking robot automatically starts cooking according to the recipe information received from the server.

[1820] Step 13:

[1821] After the cooking robot completes the cooking, it sends a cooking completion notification to the server.

[1822] Step 14:

[1823] The server receives the cooking completion notification and issues delivery instructions to the delivery service as needed.

[1824] Step 15:

[1825] A delivery service delivers pre-cooked meals to a user's location.

[1826] Step 16:

[1827] Users share leftover ingredients and cooking utensils with other users through a community platform.

[1828] Step 17:

[1829] The server stores the information posted on the community platform in a database and uses it the next time a recipe is generated.

[1830] This process dynamically generates optimal recipes based on the user's household information and sales information, and in cooperation with the cooking robot, achieves hassle-free cooking. Furthermore, the community platform also promotes the effective use of resources.

[1831] Example 1

[1832] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1833] Traditionally, when cooking at home, users had to come up with their own recipes based on the ingredients and cooking utensils they had, which was time-consuming and labor-intensive. Furthermore, there was a lack of mechanisms for efficient shopping using sales information or for sharing ingredients and cooking utensils with other users. To solve these issues, a system that provides an efficient and satisfying dining experience is needed.

[1834] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1835] In this invention, the server includes means for allowing users to input information about their home refrigerator inventory, family composition, and cooking utensils; means for saving the input information in a database; means for acquiring sales information from nearby stores; means for generating and suggesting recipes using a generative AI model; and means for transmitting the suggested recipes to the user's device. This allows users to easily obtain optimal recipes based on the ingredients, cooking utensils, and sales information available at home, enabling them to cook efficiently and save time and effort. Furthermore, sharing ingredients and cooking utensils among users promotes the reduction of food waste and the efficient use of resources.

[1836] A "user" is an entity that uses the system to provide refrigerator inventory information, family composition, and cooking utensil information.

[1837] A "database" is a storage device for storing and managing information entered by users, sale information, and the like.

[1838] "Special sale information" is information obtained from nearby stores about products being sold at discounted prices.

[1839] A "generative AI model" is an artificial intelligence algorithm that generates recipes based on accumulated data.

[1840] "Suggestion" is the act of showing a recipe generated by a generative AI model to a user.

[1841] A "terminal" is an electronic device through which a user inputs information and receives suggested recipes.

[1842] A "cooking device" is a machine that automatically cooks food based on a recipe sent from a server.

[1843] A "community platform" is an online gathering place where users can share leftover ingredients and cooking utensils.

[1844] This invention is a system that collects household information from users and generates optimal recipes based on that information. This system operates in cooperation with three entities: a server, a terminal, and the user.

[1845] Entering and collecting user information

[1846] User

[1847] Users enter information about their refrigerator inventory, household composition, and cooking appliances through an application or web portal. This information is stored on the user's device and then sent to the server.

[1848] Terminal

[1849] After receiving the information entered by the user, the device sends the data to the server, including the types and quantities of ingredients in the refrigerator, the number of family members and their food preferences, and the cooking equipment installed in the home.

[1850] server

[1851] The server stores the information sent from the device in a database that stores detailed household information for each user.

[1852] Collecting special sale information

[1853] server

[1854] The server periodically collects sales information by calling the sales information API in cooperation with nearby stores. The sales information of the stores is obtained from this API and saved in a database. This allows the latest sales information to be reflected in real time.

[1855] Recipe Generation

[1856] server

[1857] The server retrieves the user's household information and sales information from the database, and then uses the generative AI model to generate recipes based on this information. The generated recipes include optimal cooking steps and ingredient lists that take into account the current refrigerator inventory and sales information.

[1858] Recipe presentation and selection

[1859] server

[1860] The server sends the generated recipe to the user's device and presents several recipe suggestions so that the user can choose from multiple recipes.

[1861] Terminal

[1862] The terminal displays the recipe sent from the server and offers the user options.

[1863] User

[1864] The user selects the recipe that best suits their taste from the suggested recipes, and the selected information is sent back to the server via the terminal.

[1865] Automated cooking and delivery services

[1866] server

[1867] The server sends the recipe selected by the user to the cooking device and provides cooking instructions, and the cooking device automatically starts cooking based on the transmitted instructions.

[1868] cooking equipment

[1869] The cooking device automatically cooks the food according to the recipe received from the server and serves the finished dish to the user, or delivers it to the user in cooperation with a delivery service.

[1870] Community Collaboration

[1871] User

[1872] Users can share their leftover ingredients and cooking utensils with other users through the community platform, which helps reduce food waste and makes efficient use of the ingredients and cooking utensils they need.

[1873] server

[1874] The server manages transaction information between users and performs appropriate matching. This information is stored in a database and is reflected in the next recipe suggestions.

[1875] Specific examples

[1876] For example, suppose User A has 6 eggs, 500ml of milk, 3 carrots, and 200g of chicken in his refrigerator. User A also inputs that he has a family of 3 people, some of whom are vegetarian and some of whom have low-carb diets. He also owns an oven, a microwave, and a blender.

[1877] The server records in its database the sale information it has obtained from nearby stores, such as tomatoes at 100 yen per bag and beef at 500 yen per kg.

[1878] With this information, the server uses a generative AI model to generate the following recipe:

[1879] Vegetable Omelet: How to make an omelet in a frying pan using eggs, carrots, and tomatoes

[1880] Chicken and Tomato Salad: A salad made with boiled chicken, tomatoes, and lettuce.

[1881] The user selects the vegetable omelet, and the server sends this information to the cooking device, which then automatically cooks the omelet and serves it to the user at the scheduled time or delivers it via a delivery service.

[1882] In this way, the system of the present invention can make the most of each household's needs and resources to provide an efficient and satisfying dining experience.

[1883] Prompt Sentence Examples

[1884] "In the refrigerator, there are six eggs, 500ml of milk, three carrots, and 200g of chicken. There are three people in my family, and some are vegetarian and some are low-carb. I also have an oven, microwave, and blender. I'm currently seeing special offers on tomatoes for 100 yen per bag and beef for 500 yen per kg. Please generate the best recipe for me."

[1885] By inputting this prompt into a generative AI model, the optimal recipe based on the user's needs can be generated.

[1886] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1887] Step 1:

[1888] Entering user information

[1889] Users open an application or web portal and enter information about their refrigerator inventory, household composition, and cooking appliances, which is temporarily stored on the user's device.

[1890] Input: Ingredients in the refrigerator (e.g., 6 eggs, 500 ml of milk, 3 carrots, 200 g of chicken), family size (e.g., 3 people), cooking equipment (e.g., oven, microwave, blender)

[1891] Output: Temporarily stores the household information entered by the user.

[1892] Step 2:

[1893] Sending user information

[1894] The terminal transmits the information input by the user to the server. Specifically, the information is transmitted when the send button is clicked.

[1895] Input: Home information entered by the user

[1896] Output: Home information received by the server

[1897] Step 3:

[1898] Saving user information

[1899] The server stores the home information sent from the device in a database, and the stored data is organized by user.

[1900] Input: Home information sent from the device

[1901] Output: Household information stored in a database

[1902] Step 4:

[1903] Collecting special sale information

[1904] The server periodically calls the sales information API to obtain the latest sales information from nearby stores, and stores this information in a database.

[1905] Input: Call the sale information API

[1906] Output: Special sale information stored in the database

[1907] Step 5:

[1908] Recipe Generation

[1909] The server retrieves the user's household information and special sale information from the database and inputs it as a prompt into the generative AI model, which then generates the optimal recipe based on that information.

[1910] Input: Household information and special sale information obtained from the database

[1911] Output: Recipes generated by the generative AI model

[1912] Specific operation: The server generates a prompt (e.g., "In the refrigerator, there are 6 eggs, 500ml of milk, 3 carrots, and 200g of chicken. There are 3 people in the family, and some prefer vegetarian and low-carb foods. There is also an oven, microwave, and blender. There are special offers on tomatoes for 100 yen per bag and beef for 500 yen per kg. Please generate the best recipe.") and sends it to the generative AI model.

[1913] Step 6:

[1914] Submitting a Recipe

[1915] The server transmits the generated recipe to the user's terminal.

[1916] Input: Recipes generated by a generative AI model

[1917] Output: Recipe sent to terminal

[1918] Step 7:

[1919] Viewing Recipes

[1920] The terminal displays the recipe sent from the server to the user.

[1921] Input: Recipe sent from the server

[1922] Output: The recipe that is displayed to the user

[1923] Specific operation: Recipes for "Vegetable Omelette" and "Chicken and Tomato Salad" displayed on the application screen

[1924] Step 8:

[1925] Recipe Selection

[1926] The user selects a recipe from the displayed list, and the selection information is sent to the server via the terminal.

[1927] Input: User selected recipe

[1928] Output: Selected recipe information sent to the server

[1929] Step 9:

[1930] Cooking Instructions

[1931] The server transmits the recipe selected by the user to the cooking device and issues cooking instructions.

[1932] Input: Selected recipe information

[1933] Output: Cooking instructions sent to the cooking device

[1934] Step 10:

[1935] Automatic cooking

[1936] The cooking device automatically starts cooking based on the recipe information received from the server, and once the cooking is complete, it serves the food to the user or delivers it via a delivery service.

[1937] Input: Cooking instructions from the server

[1938] Output: Finished dish

[1939] Specific operation: The cooking device automatically cooks an omelet and serves the finished product to the user.

[1940] Step 11:

[1941] Community Collaboration

[1942] Users share their leftover ingredients and cooking utensils with other users through the community platform, and this sharing information is sent to the server.

[1943] Input: Information on leftover ingredients and cooking utensils shared by users

[1944] Output: Shared information sent to the server

[1945] Step 12:

[1946] Management of transaction information

[1947] The server manages transaction information between users and performs matching as appropriate. Matching information is stored in a database and reflected in the next recipe suggestions.

[1948] Input: Shared ingredients and cooking utensil information

[1949] Output: Transaction information stored in the database

[1950] Specific operation: Matching transaction information between users such as "I have two eggs left" or "I need two eggs"

[1951] Through these steps, the system generates and provides optimal recipes tailored to the needs of each household, providing an efficient and satisfying dining experience.

[1952] (Application example 1)

[1953] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1954] Conventional recipe suggestion systems were able to generate recipes based on information about the household refrigerator's inventory, household configuration, and cooking utensils, but they lacked support for users to have an efficient and effective shopping experience in a physical store. They also lacked the functionality to instantly show the generated recipe to the user and provide a shopping list on the spot. This made it difficult for users to smoothly shop and reduce the effort required for cooking.

[1955] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1956] In this invention, the server includes a means for allowing a user to input information about the inventory of a home refrigerator, family composition, and cooking equipment owned by the user, a means for storing the input information in a database, and a means for acquiring sales information from nearby stores. This allows recipes to be displayed using a smart device in a physical store. Furthermore, by adding a means for generating a shopping list based on the recipe and guiding the user through shopping in the store, the user can shop efficiently and reduce the effort required for cooking.

[1957] "Means for allowing users to input information on the inventory of their household refrigerators, their family composition, and the cooking equipment they own" refers to a method in which users input the ingredients, number of people in their household, and the cooking equipment they can use, and collect that information.

[1958] The term "means for storing the input information in a database" refers to a system or device for electronically storing and managing information input by a user.

[1959] "Means for obtaining sale information from nearby stores" refers to a method or system for obtaining sale information at a store in cooperation with the store.

[1960] "Means for suggesting recipes generated based on the stored information and the acquired sale information" refers to a method or system for generating and suggesting appropriate recipes based on stored household information and sale information.

[1961] "Means for sending the proposed recipe to a user's terminal" refers to a method or system for sending the generated recipe to an electronic terminal, such as a smartphone or tablet.

[1962] "Means for displaying the recipe using a smart device in a physical store" refers to a method or system for displaying a recipe using a smart device while the user is in the physical store.

[1963] "Means for generating a shopping list based on the recipe and guiding shopping in the store" refers to a method or system for listing ingredients required for the generated recipe and allowing efficient shopping in the store based on that list.

[1964] The present invention is a system that collects household information from users and generates optimal recipes based on that information. This system operates in cooperation with three entities: a server, a terminal, and a user.

[1965] Entering and collecting user information

[1966] User

[1967] Users input information about their home refrigerator inventory, family composition, and cooking utensils via smart devices such as smartphones, tablets, smart glasses, etc. This information is temporarily stored on the user's device and then sent to the server.

[1968] Terminal

[1969] After receiving the information entered by the user, the device sends the data to the server, including the types and quantities of ingredients in the refrigerator, the number of family members and their food preferences, and the cooking equipment installed in the home.

[1970] server

[1971] The server stores the information sent from the device in a database that stores detailed household information for each user.

[1972] Collecting special sale information

[1973] The server periodically collects sales information by calling the sales information API in cooperation with nearby stores. The sales information of the stores is obtained from this API and saved in a database. This allows the latest sales information to be reflected in real time.

[1974] Recipe Generation

[1975] The server retrieves the user's household information and sales information from the database, and then uses generative AI (e.g., OpenAI's GPT-3 model) to generate a recipe based on this information. The generated recipe includes optimal cooking steps and a list of ingredients, taking into account the current refrigerator inventory and sales information.

[1976] Recipe presentation and selection

[1977] The server sends the generated recipe to the user's smart device terminal. It presents several recipe suggestions so that the user can choose from multiple recipes. The terminal displays the recipes sent from the server and offers options to the user. The user selects the recipe that best suits their preferences from the suggested recipes. The selected information is sent back to the server via the terminal.

[1978] Displaying recipes and generating shopping lists in physical stores

[1979] The user can view the recipes using a smart device in a physical store. The smart device displays the recipes based on the user's household information and sales information. The smart device also generates a shopping list based on the generated recipes, guiding the user to efficiently shop in the store.

[1980] For example, suppose User A has 6 eggs, 500ml of milk, 3 carrots, and 200g of chicken in his refrigerator. User A also inputs that he has a family of 3 people and owns an oven, microwave, and blender. The server uses a generation AI based on sale information obtained from nearby stores to generate the following recipe:

[1981] "Vegetable Omelette": Steps for making an omelet in a frying pan using eggs, carrots, and tomatoes.

[1982] "Chicken and Tomato Salad": Steps for making a salad of boiled chicken, tomatoes, and lettuce.

[1983] An example of a generated prompt is:

[1984] "The refrigerator contains {'eggs': 6, 'milk': '500ml', 'carrot': 3, 'chicken': '200g'}. There are three people in the family. The kitchen appliances are ['oven', 'microwave', 'blender']. The sale information is ['tomatoes: 100 yen / bag', 'beef: 500 yen / kg']. Please suggest a recipe based on this."

[1985] Users can view and select recipes using their smart devices in a physical store. Based on their selection, the system generates a shopping list with the ingredients they need, allowing users to shop efficiently in the store and reduce the effort required for cooking. In this way, the system of the present invention can maximize the use of each household's needs and resources to provide an efficient and satisfying dining experience.

[1986] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1987] Step 1:

[1988] A user uses a smart device (such as a smartphone, tablet, or smart glasses) to input information about the inventory of their home refrigerator, their family composition, and the cooking appliances they own. This input information is temporarily stored in the smart device's application.

[1989] Input: refrigerator inventory information, family composition, cooking utensil information

[1990] Output: Temporary data storage (smart device)

[1991] Step 2:

[1992] The device receives the information entered by the user and transmits the data to the server, including the types and quantities of ingredients in the refrigerator, the number of family members and their food preferences, and the cooking utensils available in the home.

[1993] Input: Home information entered by the user

[1994] Output: Send data to the server

[1995] Step 3:

[1996] The server stores the information sent from the devices in a database that stores detailed household information for each individual user.

[1997] Input: Home information sent from the device

[1998] Output: Save data to database

[1999] Step 4:

[2000] The server periodically collects sales information from nearby stores and calls the sales information API. This API retrieves sales information from the stores in real time and stores it in a database.

[2001] Input: Data retrieval request from the special sale information API

[2002] Output: Save the acquired sales information in a database

[2003] Step 5:

[2004] The server retrieves the user's household information and special sale information from the database, and then generates a recipe using a generative AI model (e.g., OpenAI's GPT-3). A prompt is entered into the generative AI model to generate an appropriate recipe. An example of a prompt is as follows:

[2005] "The refrigerator contains {'eggs': 6, 'milk': '500ml', 'carrot': 3, 'chicken': '200g'}. There are three people in the family. The kitchen appliances are ['oven', 'microwave', 'blender']. The sale information is ['tomatoes: 100 yen / bag', 'beef: 500 yen / kg']. Please suggest a recipe based on this."

[2006] Input: User's home information, special offer information, prompt text

[2007] Output: Generated recipe list

[2008] Step 6:

[2009] The server sends the generated recipe to the user's smart device, where several recipes are presented to the user so that the user can choose from them.

[2010] Input: Generated recipe list

[2011] Output: Send recipe to smart device

[2012] Step 7:

[2013] The user checks the recipes presented on the smart device and selects the recipe that suits their taste. The selected recipe information is then sent back to the server via the device.

[2014] Input: Multiple recipe suggestions, user selection

[2015] Output: Selected recipe information sent to server via terminal

[2016] Step 8:

[2017] The user displays the selected recipe using a smart device in a physical store, and the smart device generates a shopping list of ingredients based on the selected recipe, helping the user to shop efficiently in the store.

[2018] Input: User selected recipe

[2019] Output: Shopping list generation and in-store shopping guidance

[2020] The above steps enable optimal recipe suggestions based on household information and special sale information, as well as efficient shopping in physical stores.

[2021] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2022] This system generates optimal recipes by taking into account the user's household information and emotional state, and performs automatic cooking in cooperation with a cooking robot. This system functions through the cooperation of four entities: a server, a terminal, a user, and an emotion engine.

[2023] Entering and collecting user information

[2024] User

[2025] Users enter information about their refrigerator inventory, household composition, and cooking appliances through a dedicated application or web portal. This information is stored on the user's device and then sent to the server.

[2026] Terminal

[2027] The device receives information entered by the user, formats it, and sends it to a server. This data includes the types and quantities of food in the refrigerator, the number of family members and their food preferences, and the cooking equipment in the home.

[2028] server

[2029] The server stores the user information sent from the terminal in a database, which stores detailed household information for each user.

[2030] Collecting special sale information

[2031] server

[2032] The server periodically collects sales information by calling the sales information API in cooperation with nearby stores. The sales information of the stores is obtained from this API and saved in a database. This allows the latest sales information to be reflected in real time.

[2033] Recipe generation and emotional engine utilization

[2034] server

[2035] The server retrieves the user information and special sale information stored in the database and passes them to the generative AI engine, which then generates the optimal recipe. The generated recipe includes a list of ingredients and cooking instructions.

[2036] Emotion Engine

[2037] The emotion engine recognizes the user's emotional state in real time and transmits the emotion data to the server, which reflects the user's detailed emotional state, such as whether they are relaxed or stressed.

[2038] server

[2039] The server optimizes the generated recipes based on the emotion data sent from the emotion engine. For example, if the user is feeling stressed, it will prioritize recipes that are quick and easy to prepare.

[2040] Recipe presentation and selection

[2041] server

[2042] The server sends the optimized recipe to the user's device and presents several options so that the user can choose from multiple recipes.

[2043] Terminal

[2044] The terminal displays the recipe sent from the server and offers the user options.

[2045] User

[2046] The user selects the recipe that best suits their taste from the suggested recipes, and the selected information is sent back to the server via the terminal.

[2047] Automated cooking and delivery services

[2048] server

[2049] The server sends the recipe selected by the user to the cooking robot and instructs it to start cooking. The cooking robot automatically starts cooking based on the instructions sent.

[2050] cooking robot

[2051] The cooking robot automatically cooks according to the recipe information received from the server, and after cooking is complete, it sends a cooking completion notification to the server.

[2052] server

[2053] The server receives the cooking completion notification and issues delivery instructions to the delivery service as needed.

[2054] Community Collaboration

[2055] User

[2056] Users can share their leftover ingredients and cooking utensils with other users through the community platform, which helps reduce food waste and makes efficient use of the ingredients and cooking utensils they need.

[2057] server

[2058] The server manages transaction information between users and performs matching as appropriate. This information is saved in a database and used the next time a recipe is generated.

[2059] Specific examples

[2060] For example, suppose User B has six eggs, 500ml of milk, three carrots, and 200g of chicken in his refrigerator. User B also inputs that he has three family members, some of whom are vegetarian and some of whom prefer low-carb meals. Furthermore, suppose he owns an oven, a microwave, and a blender. If User B's emotional state indicates "stress," the server will optimize the recipe suggestions based on this emotional data.

[2061] The server has recorded in its database sales information obtained from nearby stores that tomatoes are 100 yen per bag and beef is 500 yen per kg. Based on this information, the server uses generative AI to generate the following recipe:

[2062] Vegetable Omelette: Steps to make an omelet in a frying pan using eggs, carrots, and tomatoes.

[2063] Chicken and Tomato Salad: Steps for a salad made with boiled chicken, tomatoes, and lettuce.

[2064] Furthermore, taking into account User B's stress level, the server prioritizes an "omelette with lots of vegetables," which can be cooked in a short time. User B selects this recipe, and the server sends instructions to the cooking robot. The cooking robot automatically cooks the omelette based on these instructions and serves it to User B at the set time, or delivers it via a delivery service.

[2065] In this way, the system of the present invention can provide an efficient and satisfying dining experience by taking into maximum consideration the needs and resources of each household, as well as the user's emotional state, thereby improving QoL.

[2066] The processing flow will be explained below.

[2067] Step 1:

[2068] A user accesses an application or web portal and enters information about the household refrigerator inventory, household composition, and cooking equipment they own, such as six eggs, 500ml of milk, three carrots, and 200g of chicken.

[2069] Step 2:

[2070] The terminal receives user-entered information, formats it, and sends it to the server. Specifically, the entered information is converted into JSON format data and sent to the server using the HTTPS protocol.

[2071] Step 3:

[2072] The server stores the user information received from the device in a database, including refrigerator inventory, family composition (e.g., three people in a family, vegetarian, low-carb preference), and a list of cooking appliances (oven, microwave, blender).

[2073] sql

[2074] INSERT INTO user_data (user_id, refrigerator, family, tools)

[2075] VALUES ("user123", '{"eggs":6,"milk":"500ml","carrots":3,"chicken":"200g"}', '{"members":3,"preferences":["vegetarian","low-carb"]}', '["oven","microwave","blender"]');

[2076] Step 4:

[2077] The server calls the special sale information API to get special sale information from nearby stores, such as tomatoes for 100 yen per bag and beef for 500 yen per kg.

[2078] json

[2079] {

[2080] "special_offers": {

[2081] "tomato": "100yen / bag",

[2082] "beef": "500yen / kg"

[2083] }

[2084] }

[2085] Step 5:

[2086] The server stores the acquired sale information in a database.

[2087] sql

[2088] INSERT INTO special_offers (store_id, item, price)

[2089] VALUES ("store456", "tomato", "100yen / bag"), ("store456", "beef", "500yen / kg");

[2090] Step 6:

[2091] The server retrieves the saved user information and sale information from the database and passes it to the generation AI engine.

[2092] sql

[2093] SELECT FROM user_data WHERE user_id = "user123";

[2094] SELECT FROM special_offers WHERE store_id = "store456";

[2095] Step 7:

[2096] The generative AI engine generates optimal recipes based on the acquired data. The generated recipes include a list of ingredients and cooking instructions. For example, it generates recipes for a "vegetable omelet" and a "chicken and tomato salad."

[2097] json

[2098] {

[2099] "recipes": [

[2100] {

[2101] "name": "Vegetable Omelette",

[2102] "ingredients": ["eggs", "carrots", "tomato"],

[2103] "instructions": "Mix eggs, chop vegetables, and fry in a frying pan. Finally, pour in the eggs to make an omelet."

[2104] },

[2105] {

[2106] "name": "Chicken and Tomato Salad",

[2107] "ingredients": ["chicken", "tomato"],

[2108] "instructions": "Boil the chicken and serve it as a salad with tomatoes."

[2109] }

[2110] ]

[2111] }

[2112] Step 8:

[2113] The emotion engine recognizes the user's emotional state in real time and sends the emotion data to the server. For example, if the user is feeling stressed, the information is sent to the server.

[2114] Step 9:

[2115] The server optimizes the generated recipes based on the emotional data from the emotion engine. For example, if the user is feeling stressed, it will prioritize suggesting an "omelette with lots of vegetables," which can be cooked quickly and easily.

[2116] Step 10:

[2117] The server sends a list of optimized recipes to the user's device.

[2118] Step 11:

[2119] The device displays the recipes and offers options to the user, who can then select the recipe that best suits their taste.

[2120] Step 12:

[2121] The recipe information selected by the user is again transmitted to the server via the terminal.

[2122] Step 13:

[2123] The server sends the selected recipe to the cooking robot and instructs it to start cooking.

[2124] Step 14:

[2125] The cooking robot automatically starts cooking according to the recipe information received from the server.

[2126] Step 15:

[2127] After the cooking robot completes the cooking, it sends a cooking completion notification to the server.

[2128] Step 16:

[2129] The server receives the cooking completion notification and issues delivery instructions to the delivery service as needed.

[2130] Step 17:

[2131] A delivery service delivers pre-cooked meals to a user's location.

[2132] Step 18:

[2133] Users share leftover ingredients and cooking utensils with other users through a community platform.

[2134] Step 19:

[2135] The server stores the information posted on the community platform in a database and uses it the next time a recipe is generated.

[2136] This process dynamically and efficiently generates optimal recipes that take into account the user's household information, sales information, and emotional state, and works in conjunction with a cooking robot to achieve hassle-free cooking. Furthermore, the community platform also promotes efficient resource utilization.

[2137] Example 2

[2138] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2139] In modern households, managing refrigerator inventory and planning meals requires time and effort, while food waste is becoming a serious problem. Furthermore, recipe suggestions that take into account the user's emotional state may not be adequately tailored to the user's needs and resources. Therefore, there is a need for a system that can provide efficient and optimal meal plans that take into account food inventory information, sales information, and the user's emotional state.

[2140] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2141] In this invention, the server includes means for having a user input information on household food inventory, household configuration, and owned cooking utensils, means for organizing the input information and transmitting the input information to the server, means for saving the input information in a database, means for acquiring sales information from nearby stores, means for generating recipes using a generative AI model based on the saved information and the acquired sales information, means for recognizing the user's emotional state in real time and transmitting the data to the server, means for optimizing the generated recipe based on the emotional data, means for transmitting the optimized recipe to the user's terminal, and means for the user to select a recipe. This enables optimal recipe suggestions and automatic cooking based on the user's household information and emotional state, thereby reducing food waste and improving user satisfaction.

[2142] "Food inventory information" is data regarding the type, quantity, and condition of food stored in a household refrigerator or pantry.

[2143] "Family composition" is information about the number of family members in the user's household and their individual food preferences.

[2144] "Cooking utensils" is information about the types and number of utensils and equipment used for cooking that the user owns.

[2145] "Special Offers" is data about discounts and specially priced food items offered at nearby stores.

[2146] A "generative AI model" is an artificial intelligence algorithm that generates optimal recipes based on the user's household information and special sale information.

[2147] "Emotional state" is data that assesses a user's current emotions and stress levels in real time.

[2148] "Optimization" is the process of improving the generated recipe by taking into account the user's emotional state and other variables.

[2149] A "server" is a computer system that receives, stores, analyzes input information, and performs recipe generation and optimization.

[2150] A "terminal" is a device that a user uses to enter information, receive recipes, and make selections.

[2151] "Automatic cooking" is the process in which a cooking robot automatically prepares a dish based on a recipe instructed by the server.

[2152] A "community platform" is an online or digital system for users to share leftover ingredients and cooking equipment.

[2153] This system generates optimal recipes by taking into account the user's household information and emotional state, and performs automatic cooking in cooperation with a cooking robot. This system functions in cooperation with multiple entities: a server, a terminal, a user, and an emotion engine.

[2154] Entering and collecting user information

[2155] Users use a dedicated application or web portal to enter information about the food inventory in their refrigerator, their family composition, and the cooking equipment they own. This information is stored on the user's device and then sent to a server. The device then organizes the information entered by the user and sends it to the server. This data includes the types and quantities of ingredients in the refrigerator, the number of family members and their food preferences, and the cooking equipment installed in the home.

[2156] The server stores the user information sent from the device in a database. Detailed household information for each user is stored in the database. The server periodically collects sales information by linking with nearby stores and calling the sales information API. The sales information for each store obtained from this API is stored in the database. This allows the latest sales information to be reflected in real time.

[2157] Recipe generation and emotional engine utilization

[2158] The server retrieves saved user information and special sale information from the database and provides a prompt to the AI ​​engine based on this information. The AI ​​engine then generates the optimal recipe. For example, the prompt is "Please generate a recipe for a family of three that is vegetarian-friendly." The generated recipe includes a list of ingredients and cooking instructions.

[2159] The emotion engine recognizes the user's emotional state in real time and sends that emotional data to the server. This data reflects the user's detailed emotional state, such as whether they are relaxed or stressed. The server optimizes the generated recipes based on the emotional data sent from the emotion engine. For example, if the user is feeling stressed, it will prioritize recipes that are quick and easy to prepare.

[2160] Recipe presentation and selection

[2161] The server sends the optimized recipe to the user's device. The user is presented with several options to choose from. The device displays the recipe sent from the server and provides the user with options.

[2162] The user selects the recipe that best suits their taste from the suggested recipes, and this selection information is sent back to the server via the terminal.

[2163] Automated cooking and delivery services

[2164] The server sends the recipe selected by the user to the cooking robot and instructs it to start cooking. The cooking robot automatically starts cooking based on the instructions sent. After cooking is complete, it sends a cooking completion notification to the server.

[2165] The server receives the cooking completion notification and issues delivery instructions to the delivery service as needed.

[2166] Community Collaboration

[2167] Through the community platform, users can share their surplus ingredients and cooking utensils with other users. This reduces food waste and allows for efficient use of necessary ingredients and cooking utensils. The server manages transaction information between users and performs matching. This information is stored in a database and will be used the next time recipes are generated.

[2168] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2169] Step 1:

[2170] Users use a dedicated application or web portal to enter information about their household food inventory, household composition, and cooking equipment they own. For example, they enter information such as the number of eggs, 500ml of milk, 3 carrots, and 200g of chicken in the refrigerator. The data entered by the user is stored on the device.

[2171] Input: household food inventory information, household composition, cooking utensil information

[2172] Output: User information stored on the device

[2173] Step 2:

[2174] The device receives the information entered by the user, formats it, and sends it to the server. The data is converted to, for example, JSON format and sent to the server. Specific operations include the device organizing the user data, packetizing it in the appropriate format, and sending it to the server via the network.

[2175] Input: User information stored on the device

[2176] Output: User information sent to the server

[2177] Step 3:

[2178] The server receives the user information sent from the device and stores it in a database. Specifically, it parses the received JSON data and stores it in a structured database. This allows detailed household information for each user to be accumulated in the database.

[2179] Input: User information sent to the server

[2180] Output: User information stored in the database

[2181] Step 4:

[2182] The server periodically connects with nearby stores and calls the sale information API to collect sale information. From this API, sale information such as "Tomatoes for 100 yen / bag, beef for 500 yen / kg" is obtained. The obtained information is stored in a database.

[2183] Input: Special sale information obtained from the special sale information API

[2184] Output: Special sale information stored in the database

[2185] Step 5:

[2186] The server retrieves user information and special sale information from the database and provides a prompt to the AI ​​engine. For example, the prompt might be, "Please generate a recipe that is suitable for a family of three and suitable for vegetarians." The AI ​​engine then generates the optimal recipe based on this prompt. The generated recipe includes a list of ingredients and cooking instructions.

[2187] Input: User information and special offer information retrieved from the database, prompt text

[2188] Output: Recipe generated by the generative AI engine

[2189] Step 6:

[2190] The emotion engine recognizes the user's emotional state in real time and sends that data to the server. For example, it determines whether the user is feeling stressed and provides that information to the server. The server then optimizes the generated recipes based on the data sent from the emotion engine. Specifically, it suggests recipes that are quick and easy to prepare to users who are feeling stressed.

[2191] Input: Real-time user emotional state data

[2192] Output: Optimized recipe taking into account emotional state

[2193] Step 7:

[2194] The server generates an optimized recipe and sends it to the user's device. The device displays the received recipe and offers the user several options. For example, it presents recipes such as "vegetable omelet" and "chicken and tomato salad."

[2195] Input: Optimized recipe generated on the server

[2196] Output: Recipe choices displayed on terminal

[2197] Step 8:

[2198] The user uses the terminal to select a desired recipe from the multiple recipes presented, and this selection information is sent back to the server from the terminal.

[2199] Input: User selected recipe

[2200] Output: Selected recipe information sent to the server

[2201] Step 9:

[2202] The server sends the selected recipe information to the cooking robot and instructs it to start cooking. The cooking robot automatically cooks the food based on the instructed recipe. For example, after cooking a "vegetable omelet," it sends a cooking completion notification to the server.

[2203] Input: Selected recipe information

[2204] Output: Automatic cooking by cooking robot, notification of cooking completion

[2205] Step 10:

[2206] The server receives a cooking completion notification from the cooking robot and, based on this, issues delivery instructions to the delivery service as needed. If the user wants to receive the food at home, the server delivers it directly.

[2207] Input: Cooking completion notification

[2208] Output: Delivery instructions to the delivery service, delivery of food to the user

[2209] The above processing steps enable optimal recipe suggestions and automatic cooking that take into account the user's home information and emotional state.

[2210] (Application example 2)

[2211] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2212] In today's busy lifestyles, preparing meals at home is a time-consuming and labor-intensive task that can be stressful for users. It is particularly difficult to select efficient, cost-effective recipes while taking into account the household's ingredient inventory and the family's food preferences. Furthermore, the user's emotional state also influences meal selection, so there is a need for recipe suggestions that reduce stress and promote relaxation. The present invention solves these problems by providing a system that proposes optimal recipes that take the user's emotional state into account and enables automatic cooking.

[2213] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2214] In this invention, the server includes: means for prompting a user to input information about the user's refrigerator inventory, family composition, and cooking utensils; means for saving the input information in a database; means for acquiring sales information from nearby stores; means for suggesting recipes generated using a generative AI model based on the saved information and the acquired sales information; means for acquiring and recognizing the user's emotional state; means for optimizing the generated recipes based on the acquired emotional state; and means for transmitting the suggested recipes to the user's device and providing multiple options. This allows for the creation of recipes that reflect the user's refrigerator inventory and family food preferences, and furthermore, by taking the user's emotional state into consideration, more optimized recipe suggestions are possible. Furthermore, providing multiple options allows for flexible responses to the user's lifestyle and mood at any given time, thereby reducing the effort and stress of cooking.

[2215] A "user" is an entity that uses this system to input information about the inventory of a household refrigerator, family composition, and cooking equipment, and receives optimal recipe suggestions.

[2216] "Home refrigerator inventory information" is information about the types and quantities of food ingredients that a user keeps in their home.

[2217] "Family composition" is information about the members of the user's household, including the number of people living in the household, their age groups, food preferences, and allergy information.

[2218] "Information about cooking utensils owned" is information about the types and quantities of cooking utensils that the user owns in the home.

[2219] The "database" is an electronic information storage device for storing and managing information input by the user and acquired sale information.

[2220] "Special sale information" is information about discounts and special sales on food and ingredients obtained from nearby stores.

[2221] A "generative AI model" is an artificial intelligence algorithm that automatically generates optimal recipes based on user information and special sale information.

[2222] "User's emotional state" is information about the mental state of the user, such as whether the user is relaxed or stressed.

[2223] "Optimizing a recipe" refers to processing the generated recipe according to the user's emotional state to make more appropriate cooking suggestions.

[2224] A "user terminal" is an electronic device, such as a smartphone or tablet, through which a user inputs information and receives suggested recipes.

[2225] A "cooking robot" is a mechanical device that automatically cooks food based on a recipe selected by the user.

[2226] A "community platform" is a shared space on the Internet where users can share and exchange leftover ingredients and cooking utensils.

[2227] This system generates optimal recipes by taking into account the user's household information and emotional state, and performs automatic cooking in cooperation with food delivery and cooking robots. This system functions through the cooperation of four entities: a server, a terminal, a user, and an emotion engine.

[2228] Entering and collecting user information

[2229] User

[2230] Users enter information about their refrigerator inventory, household composition, and cooking appliances through a dedicated application or web portal. This information is stored on the user's device and then sent to the server.

[2231] Terminal

[2232] The device receives information entered by the user, formats it, and sends it to a server. This data includes the types and quantities of food in the refrigerator, the number of family members and their food preferences, and the cooking equipment in the home.

[2233] Collecting special sale information

[2234] server

[2235] The server periodically collects sales information by calling the sales information API in cooperation with nearby stores. The sales information of the stores is obtained from this API and saved in a database. This allows the latest sales information to be reflected in real time.

[2236] Recipe generation and emotional engine utilization

[2237] server

[2238] The server retrieves the user information and special sale information stored in the database and passes them to the generative AI engine, which then generates the optimal recipe. The generated recipe includes a list of ingredients and cooking instructions.

[2239] Emotion Engine

[2240] The emotion engine recognizes the user's emotional state in real time and transmits the emotion data ...

Claims

1. A means for allowing a user to input information on the inventory of a household refrigerator, family composition, and cooking utensils owned; means for storing the input information in a database; A means of obtaining sales information from nearby stores; A means for suggesting a recipe based on the stored information and the acquired sale information; means for transmitting the suggested recipe to a user's terminal; A system including:

2. The system according to claim 1 , further comprising means for transmitting a recipe selected from the suggested recipes to a cooking robot and providing cooking instructions.

3. The system according to claim 1 , further comprising means for creating a community platform for users to share surplus ingredients and cooking utensils.

Images