System

Abstract

To solve a problem in difficulty in working individuals with limited time for housework efficiently purchasing food materials tailored to their preferences and the contents in refrigerators.SOLUTION: A system comprises: means for inputting the contents in a refrigerator of a user; means for grasping the preferences of the user; means for grasping the emotional state of the user; means for proposing recipes on the basis of the grasped preferences and emotional state of the user; means for creating a shopping list on the basis of the proposed recipe; means for transmitting the created shopping list to an online supermarket; and means for providing a direct link for the user to purchase items from the online supermarket.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] One challenge facing working adults who are busy with work and don't have time to do housework is finding it difficult to efficiently purchase ingredients that suit their preferences and what's in their refrigerator. [Means for solving the problem]

[0005] To solve this problem, we provide a system that identifies the contents of a user's refrigerator and their preferences, and suggests recipes based on that. Furthermore, we create a shopping list based on the suggested recipes, and send the list to an online supermarket, providing a path for users to make purchases directly. [Brief description of the drawings]

[0006] [Figure 1]1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a process flow of the data processing system in the first embodiment of the first form example. [Figure 12] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the application example 1 of the first embodiment. [Figure 13] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the second embodiment of the second embodiment. [Figure 14] FIG. 13 is a sequence diagram showing a process flow of the data processing system in application example 2 of embodiment 2. [Figure 15] FIG. 13 is a sequence diagram showing a process flow of a data processing system in a third embodiment of the present invention. [Figure 16] FIG. 13 is a sequence diagram showing a process flow of the data processing system in the third application example of the third embodiment. [Figure 17] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] In one embodiment of the present invention, a user inputs the contents of their refrigerator into a smartphone application. The user's preferences are also understood from their past recipe selection history and from direct input. Based on this information, the system suggests recipes to the user. Based on the suggested recipes, the system automatically creates a shopping list. The created shopping list is sent to the online supermarket that the user uses, and the user can make purchases directly based on the list.

[0029] "Example 2"

[0030] As a specific example, if a user inputs that they have eggs and bacon in the fridge and selects "Japanese food" as their preference, the system will suggest a recipe for "Japanese-style scrambled eggs with eggs and bacon." The system will then add the seasonings and accompanying vegetables needed for that recipe to a shopping list, which will be sent to the online supermarket the user uses. The user can then use that list to purchase the ingredients they need all at once.

[0031] The process flow of each embodiment will be described below.

[0032] "Example 1"

[0033] Step 1: The user opens the application on their smartphone and enters the contents of their refrigerator.

[0034] Step 2: The user inputs their preferences (e.g. Japanese, Western, Chinese, etc.) into the application.

[0035] Step 3: The system suggests recipes based on what's in the user's fridge and their preferences.

[0036] Step 4: The system automatically creates a shopping list based on the suggested recipes.

[0037] Step 5: The system sends the shopping list created by the user to the online supermarket that the user uses.

[0038] Step 6: The user purchases ingredients from the online supermarket based on the shopping list.

[0039] "Example 2"

[0040] Step 1: The user tells the application that they have eggs and bacon in the fridge.

[0041] Step 2: The user enters "Japanese food" as a preference into the application.

[0042] Step 3: The system suggests a recipe for "Japanese-style scrambled eggs with eggs and bacon."

[0043] Step 4: The system adds the condiments and vegetables needed for the recipe to a shopping list.

[0044] Step 5: The system sends the shopping list created by the user to the online supermarket that the user uses.

[0045] Step 6: The user purchases ingredients from the online supermarket based on the shopping list.

[0046] Example 1

[0047] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0048] In conventional systems, the process of users managing the contents of their refrigerator, finding recipes that matched their tastes, and purchasing the necessary ingredients was cumbersome. In addition, users would create shopping lists without considering the ingredients they already had, which could result in unnecessary purchases. Furthermore, the purchasing process at online supermarkets was done manually, which was time-consuming and cumbersome.

[0049] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0050] In this invention, the server includes a means for a user to input the contents of the refrigerator, a means for grasping the user's preferences, a means for proposing recipes using a generative AI model based on the grasped information, a means for automatically creating a shopping list based on the proposed recipes, a means for transmitting the created shopping list to an online supermarket, and a means for providing a path for the user to directly purchase at the online supermarket. This enables the user to efficiently manage the contents of the refrigerator, easily find recipes that suit their preferences, and purchase the necessary ingredients without waste.

[0051] "Means for users to input the contents of their refrigerator" refers to an interface that allows users to input the ingredients in their refrigerator using a smartphone or other device.

[0052] "Means of understanding user preferences" refers to a function for collecting and storing a user's food preferences based on the user's past recipe selection history or direct input.

[0053] "Means of suggesting recipes using a generative AI model" refers to a function that uses a generative AI model to generate and suggest appropriate recipes based on data on the contents of the user's refrigerator and their preferences.

[0054] The "means for automatically generating a shopping list" is a function for automatically generating a list of ingredients needed based on a suggested recipe.

[0055] "Means for sending a shopping list to an online supermarket" is a function for sending the created shopping list to the online supermarket used by the user.

[0056] "Means of providing a path for users to make purchases directly from online supermarkets" refers to a function that enables users to purchase groceries directly on the online supermarket platform based on the shopping list submitted to them.

[0057] The present invention is a system that allows a user to manage the contents of a refrigerator, suggest recipes that suit the user's preferences, and purchase the necessary ingredients efficiently. A specific embodiment of this system will be described below.

[0058] System Configuration

[0059] Hardware and Software

[0060] User terminal: Mobile device such as a smartphone or tablet. Uses applications that run on iOS and ANDROID (registered trademark).

[0061] Server: Located on the cloud, it hosts the database and generative AI model. It uses MySQL (registered trademark) for the database and OpenAI (registered trademark)'s GPT-4 (registered trademark) for the generative AI model.

[0062] Entering and Saving Data

[0063] The user enters the contents of the refrigerator

[0064] A user opens the smartphone application and inputs the ingredients in the refrigerator, either manually, by voice, by scanning a barcode, etc. For example, a user inputs "chicken, carrots, onions."

[0065] Understand user preferences

[0066] The user's preferences are captured from their past recipe selection history or through direct input. For example, if a user has previously selected "curry" as their favorite dish, that information is also stored in the database.

[0067] Recipe Suggestions

[0068] Server suggests recipes

[0069] The server uses a generative AI model to suggest recipes based on the contents of the user's fridge and their preferences. Specifically, it sends the following prompt to the generative AI model:

[0070] Contents of user's fridge: Chicken, carrots, onions

[0071] User preference: Curry

[0072] Generate suggested recipes.

[0073] The generative AI model generates a recipe for "chicken curry" based on this prompt, and the server sends the recipe to the user's smartphone.

[0074] Create and send a shopping list

[0075] The server creates the shopping list.

[0076] When a user selects a suggested recipe, the server automatically creates a shopping list of ingredients needed for that recipe, for example, "chicken, carrots, onions, curry roux, and any other necessary seasonings."

[0077] The server sends the shopping list to the online supermarket.

[0078] The server then sends the created shopping list to the online supermarket the user uses, using an API to link with the online supermarket's system.

[0079] Making a purchase

[0080] The user purchases from the online supermarket

[0081] The user opens the online supermarket's application or website, checks the shopping list sent from the server, and can then directly purchase the ingredients he or she needs based on the list.

[0082] Examples

[0083] For example, if a user inputs that they have "chicken, carrots, and onions" in their fridge and has previously preferred "curry," the system will suggest a recipe for "chicken curry." Here is an example of a prompt to be input to the generative AI model:

[0084] Contents of user's fridge: Chicken, carrots, onions

[0085] User preference: Curry

[0086] Generate suggested recipes.

[0087] Based on this prompt, the generative AI model generates a recipe for "chicken curry" and suggests it to the user. Based on the suggested recipe, the system creates a shopping list of "chicken, carrots, onions, curry roux, and other necessary seasonings" and sends it to the online supermarket. The user can use this list to directly purchase the necessary ingredients.

[0088] In this way, users can efficiently manage the contents of their refrigerator, easily find recipes that suit their tastes, and buy the ingredients they need without waste.

[0089] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0090] Step 1:

[0091] The user enters the contents of the refrigerator

[0092] The user opens the smartphone application and inputs the ingredients in the refrigerator. Input methods include manual input, voice input, barcode scanning, etc. For example, the user inputs "chicken, carrots, onions." This input data is stored in a database on the cloud. The input data includes the names of ingredients and their quantities.

[0093] Step 2:

[0094] Understand user preferences

[0095] A user's preferences are understood from past recipe selection history or direct input. For example, if a user has previously selected "curry" as their favorite food, that information is also stored in the database. The server analyzes the user's selection history and extracts preference trends. The input data is the past recipe selection history or direct input of the user's preferences, and the output data is the user's preference trends.

[0096] Step 3:

[0097] Server suggests recipes

[0098] The server uses a generative AI model to suggest recipes based on the contents of the user's fridge and their preferences. Specifically, it sends the following prompt to the generative AI model:

[0099] Contents of user's fridge: Chicken, carrots, onions

[0100] User preference: Curry

[0101] Generate suggested recipes.

[0102] The generative AI model generates a recipe for "chicken curry" based on this prompt, and the server sends the recipe to the user's smartphone. The input data is the contents of the refrigerator and the user's preferences, and the output data is the suggested recipe.

[0103] Step 4:

[0104] The server creates the shopping list.

[0105] When a user selects a suggested recipe, the server automatically creates a shopping list of ingredients needed based on the recipe. For example, it generates a list of "chicken, carrots, onions, curry roux, and other necessary seasonings." The server also provides a function to remove ingredients from the list that the user already has. The input data is the selected recipe and the contents of the refrigerator, and the output data is the shopping list.

[0106] Step 5:

[0107] The server sends the shopping list to the online supermarket.

[0108] The server sends the created shopping list to the online supermarket that the user uses. An API is used to send the list, linking with the online supermarket's system. The input data is the shopping list and the online supermarket's API information, and the output data is the shopping list sent to the online supermarket.

[0109] Step 6:

[0110] The user purchases from the online supermarket

[0111] The user opens the online supermarket application or website and checks the shopping list sent from the server. The user can directly purchase the ingredients he needs based on the list. The input data is the shopping list sent to the online supermarket, and the output data is the purchased ingredients.

[0112] (Application example 1)

[0113] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0114] Conventional refrigerator management systems had the problem that users had to manually manage the contents of their refrigerators, search for recipes themselves, and list the ingredients they needed to purchase, which was a lot of work. In addition, it was difficult to suggest recipes that matched the user's preferences, and creating shopping lists and purchasing procedures were cumbersome. This reduced user convenience and made efficient ingredient management difficult.

[0115] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0116] In this invention, the server includes a means for grasping the contents of the user's refrigerator and each user's preferences, a means for proposing recipes based on the grasped information, a means for creating a shopping list according to the proposed recipes, and a means for transmitting the created shopping list to an electronic commerce platform and providing a path for the user to make direct purchases. This enables the user to efficiently manage the contents of the refrigerator, receive recipe suggestions according to their preferences, and easily purchase the necessary ingredients.

[0117] A "user" is an individual who uses the system to input the contents of their refrigerator and their preferences and receive recipe suggestions and shopping lists.

[0118] "What's in the refrigerator" refers to information about the food and beverages currently stored in the user's refrigerator.

[0119] "Preferences" are the ingredients and cooking preferences that a user indicates based on their past recipe selection history or direct input.

[0120] A "recipe" is a list of instructions and ingredients needed to make a particular dish.

[0121] A "shopping list" is a list of ingredients and products to purchase that is created based on a suggested recipe.

[0122] An "e-commerce platform" is an online marketplace for purchasing goods and services via the Internet.

[0123] A "guideline" is the sequence of steps or operations that allow a user to directly purchase items based on their shopping list.

[0124] A "generative AI model" is an artificial intelligence algorithm used to suggest optimal recipes based on what's in a user's fridge and their preferences.

[0125] A "prompt sentence" is an instruction sentence for suggesting recipes or creating a shopping list that is generated by a generative AI model based on information input by the user.

[0126] A system for implementing the present invention includes means for identifying the contents of a user's refrigerator and each user's preferences, means for suggesting recipes based on the identified information, means for creating a shopping list based on the suggested recipes, and means for transmitting the created shopping list to an e-commerce platform and providing a path for the user to make direct purchases.

[0127] System Programming

[0128] The server provides an interface for users to input the contents of their refrigerator using their smartphones. The user's preferences are understood from their past recipe selection history and direct input. This information is sent to the server and stored in a database.

[0129] The server uses a generative AI model to suggest optimal recipes based on the contents of the user's refrigerator and their preferences. The generative AI model generates prompts based on the user's input information and suggests recipes.

[0130] Based on the suggested recipes, the server automatically creates a shopping list, which is then sent to the e-commerce platform for users to purchase directly.

[0131] Hardware and Software Used

[0132] Hardware: Smartphones, servers

[0133] Software: Python, Requests library, generative AI models

[0134] Data processing and calculation

[0135] 1. User inputs the contents of the refrigerator: The user inputs the contents of the refrigerator using a smartphone application. This information is sent to the server and stored in a database.

[0136] 2. Understanding user preferences: User preferences are understood from past recipe selection history and direct input. This information is also sent to the server and stored in the database.

[0137] 3. Recipe Suggestion: The server uses a generative AI model to suggest optimal recipes based on the contents of the user's refrigerator and their preferences. The generative AI model generates prompts based on the user's input information and makes recipe suggestions.

[0138] 4. Create shopping list: Based on the suggested recipes, the server automatically creates a shopping list.

[0139] 5. Send shopping list: Once created, the shopping list is sent to the e-commerce platform to allow the user to make purchases directly.

[0140] Examples

[0141] Suppose a user inputs that they have “Ingredient A” and “Ingredient B” in their fridge, and that they prefer “vegetarian” and “non-spicy” food. If they have previously selected “Recipe A” and “Recipe B,” the system will suggest “Recipe C” and “Recipe D.” It will then generate a shopping list that includes “Ingredient 1” and “Ingredient 2” and send it to the e-commerce platform.

[0142] Examples of prompt statements

[0143] User ID: 12345

[0144] Contents of the fridge: ["ingredient A", "ingredient B"]

[0145] User Preferences: {"Vegetarian": True, "Spicy": False}

[0146] Past recipe selection history: ["Recipe A", "Recipe B"]

[0147] Suggested recipes: ["Recipe C", "Recipe D"]

[0148] Generated shopping list: ["ingredient 1", "ingredient 2"]

[0149] Sending status to online supermarket: 200

[0150] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0151] Step 1:

[0152] The user inputs the contents of the refrigerator using a smartphone application.

[0153] Input: Information about ingredients in the user's refrigerator (e.g., ingredient A, ingredient B)

[0154] Data processing: The entered ingredient information is converted into JSON format and sent to the server.

[0155] Output: The contents of the fridge sent to the server.

[0156] Step 2:

[0157] The server knows the user's preferences.

[0158] Input: User's past recipe selection history and direct preference input (e.g. vegetarian, not spicy)

[0159] Data processing: Store user preference information in a database.

[0160] Output: User preferences stored in a database

[0161] Step 3:

[0162] The server suggests recipes using a generative AI model.

[0163] Input: Fridge contents data, user preferences data

[0164] Data processing: A generative AI model uses this data to generate prompts and suggest optimal recipes.

[0165] Output: Suggested recipes (e.g. Recipe C, Recipe D)

[0166] Step 4:

[0167] The server creates a shopping list based on the suggested recipes.

[0168] Input: Suggested recipe data

[0169] Data processing: List ingredients needed for recipes and generate shopping lists.

[0170] Output: The created shopping list (e.g. ingredient 1, ingredient 2)

[0171] Step 5:

[0172] The server transmits the created shopping list to the e-commerce platform.

[0173] Input: The shopping list data created

[0174] Data processing: Formatting the shopping list to send to the e-commerce platform's API.

[0175] Output: Status of submission to e-commerce platform (e.g. 200 OK)

[0176] Step 6:

[0177] A user purchases products based on a shopping list through an e-commerce platform.

[0178] Input: Shopping list submitted to e-commerce platform

[0179] Data processing: The e-commerce platform searches for products based on the shopping list and completes the purchase.

[0180] Output: A confirmation message for the purchase (e.g. Purchase Completed)

[0181] Example 2

[0182] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0183] Conventional recipe suggestion systems would only suggest appropriate recipes when users input the ingredients they had in their fridge and their preferred cooking genre, but lacked the functionality to automatically list the necessary seasonings and accompaniments and send them to the online supermarket. As a result, users had to manually create a shopping list based on the suggested recipes and then go through the purchasing process separately at the online supermarket. This process was time-consuming and inefficient.

[0184] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0185] In this invention, the server includes a means for grasping the contents of a user's refrigerator and each user's preferences, a means for proposing a recipe using a generative AI model based on the grasped information, a means for creating a shopping list including necessary seasonings and accompaniment ingredients based on the proposed recipe, a means for transmitting the created shopping list to an online supermarket, and a means for providing a path for the user to purchase the necessary ingredients in one go through the online supermarket. As a result, the user is automatically suggested appropriate recipes based on the contents of the refrigerator and their preferences, and is able to efficiently purchase the necessary ingredients.

[0186] "User" refers to an individual who uses the system to input the contents of their refrigerator and their preferred cuisine genres.

[0187] "What's in the refrigerator" refers to the food items currently stored in the user's refrigerator.

[0188] "Favorite Cuisine Genre" refers to a particular style or type of cuisine selected by a User.

[0189] A "generative AI model" refers to an artificial intelligence algorithm that generates an appropriate recipe based on input information.

[0190] A "prompt sentence" refers to an input sentence used to request a generative AI model to generate a recipe.

[0191] A "recipe" is a list of instructions or ingredients for making a particular dish.

[0192] "Condiments" refer to ingredients used to adjust the flavor of food.

[0193] "Accompaniment ingredients" refers to additional ingredients served with the main dish.

[0194] A "shopping list" refers to a list of ingredients needed based on a suggested recipe.

[0195] An "online supermarket" refers to a website or application that provides the service of purchasing food and daily necessities online.

[0196] "Path" refers to the steps and flow that users take to purchase the ingredients they need in one go through an online supermarket.

[0197] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0198] This invention is a system that, when a user inputs the ingredients in the refrigerator and the type of cooking they like, suggests appropriate recipes, automatically creates a shopping list including the necessary seasonings and garnishes, and transmits it to an online supermarket. A specific embodiment of this system is described below.

[0199] 1. System Overview

[0200] The system involves a series of processes: identifying the contents of a user's refrigerator and preferred cooking genres, suggesting recipes using a generative AI model, creating a shopping list based on the suggested recipes, and sending it to the online supermarket.

[0201] 2. Hardware and Software Used

[0202] Hardware: The device used by the user (smartphone, tablet, computer, etc.)

[0203] Software: Server, generative AI model (e.g. OpenAI's GPT-4), online supermarket API

[0204] 3. Program Processing

[0205] 1. Receiving User Input

[0206] The user inputs into the device the ingredients they have in their fridge (e.g. eggs and bacon) and their preferred cuisine type (e.g. Japanese food).

[0207] The terminal transmits the input information to the server.

[0208] 2. Recipe suggestions

[0209] The server analyzes the received information on ingredients and cuisine genre.

[0210] The server sends prompts to the generative AI model, which then generates an appropriate recipe.

[0211] The server sends the generated recipe (e.g. "Japanese-style scrambled eggs with eggs and bacon") back to the terminal.

[0212] 3. Generate a shopping list

[0213] The server analyzes the generated recipe and lists the necessary condiments and side dishes.

[0214] The server sends this shopping list to the API of the online supermarket the user is using.

[0215] 4. Purchase procedure

[0216] The user accesses the online supermarket's website through their device and checks the submitted shopping list.

[0217] Users purchase the ingredients they need in bulk.

[0218] 4. Specific Examples

[0219] The user types into the device, "I have eggs and bacon in the fridge," and selects "Japanese food" as their preferred cuisine genre.

[0220] The server uses a generative AI model to suggest a recipe for "Japanese-style scrambled eggs with eggs and bacon."

[0221] The server adds the necessary seasonings (e.g. soy sauce, mirin) and accompanying vegetables (e.g. spinach) to a shopping list and sends it to the online supermarket.

[0222] Users check their shopping list on the online supermarket website and purchase all the ingredients they need in one go.

[0223] 5. Examples of prompts

[0224] I have eggs and bacon in the fridge. Can you suggest some Japanese recipes?

[0225] The above is a specific embodiment for carrying out the present invention. This system automatically suggests suitable recipes to users based on the contents of their refrigerator and their preferences, enabling them to efficiently purchase the ingredients they need.

[0226] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0227] Program processing flow

[0228] Step 1: Receiving User Input

[0229] The user inputs into the device the ingredients they have in their fridge (e.g. eggs and bacon) and their preferred cuisine type (e.g. Japanese food).

[0230] Input: A user types "eggs and bacon" into a terminal application and selects "Japanese food" from a drop-down menu.

[0231] The terminal transmits the input information to the server.

[0232] Output: The data sent from the terminal to the server is ingredient information and cuisine genre information.

[0233] Specific actions: A user opens an application on their smartphone, types "eggs and bacon" into the text box, selects "Japanese food" from the drop-down menu, and presses the "Submit" button.

[0234] Step 2: Suggest a recipe

[0235] The server analyzes the received information on ingredients and cuisine genre.

[0236] Input: Ingredient information and cuisine genre information received by the server from the terminal.

[0237] The server sends prompts to the generative AI model, which then generates an appropriate recipe.

[0238] Data processing: The server sends a prompt to the generative AI model: "There are eggs and bacon in the fridge. Please suggest a Japanese recipe."

[0239] Output: Recipe information returned from the generative AI model.

[0240] The server sends the generated recipe (e.g. "Japanese-style scrambled eggs with eggs and bacon") back to the terminal.

[0241] Specific operation: The server sends a prompt to the generative AI model and displays the returned recipe on the user's device.

[0242] Step 3: Generate a shopping list

[0243] The server analyzes the generated recipe and lists the necessary condiments and side dishes.

[0244] Input: The generated recipe information.

[0245] Data processing: The server extracts the necessary seasonings (e.g. soy sauce, mirin) and accompanying vegetables (e.g. spinach) from the recipe.

[0246] Output: A list of ingredients needed.

[0247] The server sends this shopping list to the API of the online supermarket the user is using.

[0248] Specific behavior: The server sends a request to the online supermarket's API saying, "Please add these items to your shopping list."

[0249] Step 4: Checkout

[0250] The user accesses the online supermarket's website through their device and checks the submitted shopping list.

[0251] Input: The shopping list sent by the server.

[0252] Output: Shopping list displayed on the online supermarket website.

[0253] Users purchase the ingredients they need in bulk.

[0254] Specific behavior: The user opens the online supermarket app, goes to the "Shopping List" section, checks the list sent from the server, and presses the "Purchase" button.

[0255] The above is the specific processing flow of the program of this system.

[0256] (Application example 2)

[0257] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0258] Conventional recipe suggestion systems were insufficient in suggesting recipes that took into account the ingredients in the user's refrigerator and preferences, and had problems with creating a shopping list based on the suggested recipe and the complicated process of purchasing the necessary ingredients. In addition, there was no way for users to easily purchase additional ingredients required for the suggested recipe, which resulted in low convenience. There is a need to solve these problems and improve user convenience.

[0259] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0260] In this invention, the server includes a means for grasping the contents of a user's refrigerator and each user's preferences, a means for proposing recipes using a generative AI model based on the grasped information, a means for creating a shopping list according to the proposed recipes, and a means for transmitting the created shopping list to a food delivery service and providing a path for the user to directly purchase. This allows the user to receive recipe suggestions based on the ingredients in the refrigerator and their preferences, and further allows the user to easily purchase any additional ingredients that are required.

[0261] A "user" is an individual who uses the system to input the contents of their refrigerator and their preferences and receive recipe suggestions and shopping lists.

[0262] "Contents of the refrigerator" refers to ingredients and foods currently stored in the user's refrigerator.

[0263] "Preferences" refers to a user's personal preferences for a particular cuisine genre or seasoning.

[0264] A "generative AI model" is an artificial intelligence model that generates appropriate recipes based on user input information.

[0265] A "recipe" is a set of instructions that lists the steps and ingredients needed to make a particular dish.

[0266] A "shopping list" is a list of ingredients and seasonings needed based on a suggested recipe.

[0267] A "food delivery service" is a service that delivers ingredients and meals ordered online by users to their homes.

[0268] A "guideline" refers to the flow of procedures and operations that allow users to easily purchase the ingredients they need through the system.

[0269] A system for implementing this invention includes means for understanding the contents of a user's refrigerator and their individual preferences, means for suggesting recipes using a generative AI model based on the understood information, means for creating a shopping list based on the suggested recipes, and means for sending the created shopping list to a food delivery service and providing a path for the user to make direct purchases.

[0270] Hardware and Software Configuration

[0271] Hardware: Smartphones, servers

[0272] Software: Python, OpenAI API, HTTP request library (requests)

[0273] Data processing and calculation

[0274] 1. Receiving user input: The user uses their smartphone to input the ingredients they have in their fridge and their preferred cuisine, for example, "eggs, bacon" and "Japanese food."

[0275] 2. Using a generative AI model: The server uses the OpenAI API to generate a recipe based on the user's input. An example of a prompt is as follows:

[0276] Ingredients in the fridge: eggs, bacon

[0277] Favorite food genre: Japanese food

[0278] Suggested recipe:

[0279] 3. Creating a shopping list: The server extracts the additional ingredients required from the generated recipe and creates a shopping list. For example, "soy sauce, mirin, and green onion" are added.

[0280] 4. Sending an order to a food delivery service: The server sends the created shopping list to the food delivery service and provides a path for the user to purchase directly. This allows the user to easily purchase the ingredients they need.

[0281] Examples

[0282] If a user inputs that they have "eggs and bacon" in their fridge and selects "Japanese food" as their preferred cuisine genre, the server will use the generative AI model to suggest a recipe for "Japanese-style scrambled eggs with eggs and bacon." It will then add "soy sauce, mirin, and green onions" as additional ingredients needed to a shopping list and send this list to a food delivery service. The user can easily purchase these ingredients and have them delivered via their smartphone.

[0283] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0284] Step 1:

[0285] The user uses a smartphone to input the ingredients in the refrigerator and the type of cuisine they prefer.

[0286] Input: Ingredients in the fridge (e.g., "eggs, bacon"), favorite cuisine (e.g., "Japanese food")

[0287] Output: User input data

[0288] Specific actions: A user opens the application on their smartphone, enters ingredients in the fridge into a text box, and selects their preferred cuisine from a drop-down menu.

[0289] Step 2:

[0290] The server receives the user input data and generates the recipe using a generative AI model.

[0291] Input: User-entered data (ingredients in the fridge, favorite cuisine genres)

[0292] Output: Suggested recipe

[0293] Specific operation: The server calls the OpenAI API and sends the following prompt sentence to the generative AI model.

[0294] Ingredients in the fridge: eggs, bacon

[0295] Favorite food genre: Japanese food

[0296] Suggested recipe:

[0297] The generative AI model generates a recipe based on this prompt and returns it to the server.

[0298] Step 3:

[0299] The server extracts any additional ingredients required from the generated recipe and creates a shopping list.

[0300] Input: Suggested recipe

[0301] Output: Shopping list

[0302] Specific behavior: The server parses the generated recipe text and extracts any additional ingredients needed (e.g., "soy sauce, mirin, green onion"), organizes these ingredients into a list, and creates a shopping list.

[0303] Step 4:

[0304] The server creates a shopping list and sends it to the food delivery service, providing a path for the user to make a purchase directly.

[0305] Input: Shopping list

[0306] Output: Order data for food delivery service

[0307] What happens: The server calls the food delivery service's API and sends the shopping list as order data, including the user's address information, to ensure that the order is processed successfully.

[0308] Step 5:

[0309] The user receives an order confirmation from the food delivery service via smartphone and purchases the necessary ingredients.

[0310] Input: Order confirmation from food delivery service

[0311] Output: Purchased ingredients

[0312] How it works: The user receives an order confirmation on their smartphone app, waits for the delivery of the ingredients they need, and then collects the purchased ingredients when the delivery arrives.

[0313] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0314] "Example 1"

[0315] The present invention is a system including a means for grasping the contents of a user's refrigerator and each user's preferences, a means for suggesting recipes based on the grasped information, a means for creating a shopping list based on the suggested recipes, and an emotion engine for recognizing the user's emotions. The emotion engine has a function for adjusting the recipe suggestions and the shopping list according to the user's emotions. For example, when the user is feeling stressed, the emotion engine suggests recipes that include ingredients that have a relaxing effect. Also, when the user is feeling happy, the emotion engine suggests special recipes suitable for celebrations. This makes it possible to suggest meals according to the user's emotional state.

[0316] "Example 2"

[0317] As a concrete example, consider the case where a user inputs to the application that they have eggs and bacon in the fridge and selects "Japanese food" as their preference. Furthermore, suppose the user inputs to the emotion engine that they "feel stressed." In this case, the system will suggest a recipe for "Japanese-style scrambled eggs with eggs and bacon" and create a shopping list that adds "chamomile tea," which has a relaxing effect. This makes it possible to suggest meals according to the user's emotional state.

[0318] The process flow of each embodiment will be described below.

[0319] "Example 1"

[0320] Step 1: The user enters the contents of the fridge and each person's preferences into the application.

[0321] Step 2: The system will understand the information you entered and suggest recipes based on it.

[0322] Step 3: The system creates a shopping list with suggested recipes.

[0323] Step 4: The user inputs their current emotional state into the emotion engine.

[0324] Step 5: The emotion engine adapts recipe suggestions and shopping lists based on the user's emotions.

[0325] "Example 2"

[0326] Step 1: The user tells the application what eggs and bacon they have in the fridge, and that their preference is "Japanese food."

[0327] Step 2: The user types "I'm stressed" into the emotion engine.

[0328] Step 3: The system understands the input information and emotional state and suggests a recipe for "Japanese-style scrambled eggs with eggs and bacon."

[0329] Step 4: The system creates a shopping list to which it adds the relaxing "chamomile tea."

[0330] Example 1

[0331] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0332] Conventional recipe suggestion systems have problems in that it is difficult to suggest recipes that fully take into account the contents of the user's refrigerator and preferences, and they are unable to suggest recipes or adjust shopping lists according to the user's emotional state. Furthermore, creating a shopping list based on the suggested recipes and sending it to the online supermarket is done manually, which is time-consuming for users.

[0333] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0334] In this invention, the server includes means for inputting the contents of the user's refrigerator, means for grasping the user's preferences, means for suggesting recipes based on the grasped information, means for automatically creating a shopping list according to the suggested recipes, means for transmitting the created shopping list to the online supermarket, and means for recognizing the user's emotional state and adjusting the recipe suggestions and shopping list accordingly. This makes it possible to automatically create optimal recipe suggestions and shopping lists according to the contents of the user's refrigerator, preferences, and emotional state, and to transmit them to the online supermarket.

[0335] "Means for inputting the contents of the user's refrigerator" refers to an interface that allows a user to input the ingredients in their refrigerator via a smartphone or other device.

[0336] The "means of understanding user preferences" refers to a system that stores in a database the recipes the user has previously chosen, as well as the ingredients and types of dishes that the user has registered as their preferences, in order to understand the user's tastes.

[0337] The "means of suggesting recipes based on the captured information" refers to an algorithm that searches for and suggests appropriate recipes based on the contents of the user's refrigerator and their preferences.

[0338] The "means for automatically creating a shopping list according to a proposed recipe" is a system that lists ingredients needed for a proposed recipe and automatically generates a shopping list.

[0339] "Means for sending the created shopping list to the online supermarket" refers to a system that sends the created shopping list via the API of the online supermarket used by the user, enabling the user to make purchases directly.

[0340] "Means for recognizing the user's emotional state and adjusting recipe suggestions and shopping lists accordingly" refers to a system that recognizes the user's emotional state through sensors and input data, and suggests and adjusts recipes and shopping lists according to that state.

[0341] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0342] This invention is a system that proposes optimal recipes based on the contents of a user's refrigerator, preferences, and emotional state, and automatically creates and transmits a shopping list. A specific embodiment of this system will be described below.

[0343] Input the contents of the user's refrigerator

[0344] Users use a smartphone application to input the contents of their refrigerator. The application runs on iOS and Android platforms. Users enter ingredients such as "chicken, cabbage, and carrots" by text into the application's input form, or scan the barcodes of the ingredients using the barcode scanner function. The input data is sent from the device to the server and stored in a database.

[0345] Understanding user preferences

[0346] Users input recipes they have previously chosen within the application, as well as ingredients and types of cuisine they have registered as preferences. For example, if a user inputs that they "like Japanese food," that information is sent from the device to the server and stored in the user preference database. The server uses this data to determine the user's preferences.

[0347] Recipe Suggestions

[0348] The server suggests appropriate recipes based on the contents of the refrigerator and the user's preferences, using an algorithm that references the recipe database and the user's preference database. For example, if a user has "chicken, cabbage, and carrots," the server will suggest "stir-fried chicken and cabbage" or "teriyaki chicken."

[0349] Automatic shopping list creation

[0350] The server automatically creates a shopping list based on the proposed recipe. Specifically, the ingredients required for the recipe are added to the list. For example, if a recipe for "stir-fried chicken with vegetables" is proposed, the necessary ingredients (chicken, cabbage, carrots, seasonings, etc.) are added to the list. This list is sent to the user's device and displayed within the application.

[0351] Send to online supermarket

[0352] When a user checks the shopping list and wishes to purchase from an online supermarket, a request is sent from the terminal to the server. The server sends the shopping list through the API of the online supermarket the user is using. For example, the list is sent to a service such as an online shopping platform. The user can then make a purchase directly based on the list.

[0353] Emotion engine adjustment

[0354] The server recognizes the user's emotional state using the emotion engine. Specifically, the user inputs their emotional state within the application, or detects emotions through the smartphone's sensors (camera and microphone). For example, if the user inputs that they are "feeling stressed," the emotion engine will suggest recipes that contain ingredients that have a relaxing effect. Also, if the user inputs that they are "feeling happy," the emotion engine will suggest special recipes suitable for celebrations.

[0355] Examples of concrete examples and prompts

[0356] If a user inputs that they have "chicken, cabbage, and carrots" in their fridge and has previously registered that they like "Japanese food," the server will suggest "chicken and cabbage stir fry" or "teriyaki chicken." If the emotion engine recognizes that the user is stressed, it will also suggest "relaxing herbal tea."

[0357] Example prompt:

[0358] "I have some chicken, cabbage, and carrots in the fridge. I like Japanese food. I'm feeling stressed. Can you recommend some recipes?"

[0359] In this way, based on the contents of the user's fridge, preferences, and emotional state, the server suggests optimal recipes and shopping lists.

[0360] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0361] Step 1:

[0362] The user launches the smartphone application and inputs the contents of the refrigerator. Input methods include text input and barcode scanner functions. When the user inputs "chicken, cabbage, carrots," the data is sent from the device to the server. The server stores the received data in a database. The input data is a list of ingredients, and the output is the ingredient information stored in the database.

[0363] Step 2:

[0364] Within the application, the user inputs previously selected recipes, favorite ingredients, and types of cuisine. For example, if the user inputs "I like Japanese food," that information is sent from the device to the server and stored in a user preference database. The server uses this data to understand the user's preferences. The input data is the user's preference information, and the output is the preference information stored in the database.

[0365] Step 3:

[0366] The server suggests appropriate recipes based on the contents of the refrigerator and preference data entered by the user. Specifically, the server searches the recipe database and extracts recipes that match the contents of the user's refrigerator. For example, if the user has "chicken, cabbage, and carrots," the server will suggest "stir-fried chicken and cabbage" or "teriyaki chicken." The input data are the contents of the refrigerator and preference information, and the output is the suggested recipe.

[0367] Step 4:

[0368] The server automatically creates a shopping list based on the proposed recipe. Specifically, the ingredients required for the recipe are added to the list. For example, if a recipe for "stir-fried chicken with vegetables" is proposed, the necessary ingredients (chicken, cabbage, carrots, seasonings, etc.) are added to the list. This list is sent to the user's device and displayed within the application. The input data is the proposed recipe, and the output is the shopping list.

[0369] Step 5:

[0370] If the user checks the shopping list and wishes to make a purchase at the online supermarket, a request is sent from the terminal to the server. The server sends the shopping list through the API of the online supermarket the user is using. For example, the list is sent to a service such as an online shopping platform. The user can then make a purchase directly based on the list. The input data is the shopping list, and the output is the result of sending it to the online supermarket.

[0371] Step 6:

[0372] The server recognizes the user's emotional state using the emotion engine. Specifically, the user inputs their emotional state within the application or detects emotions through the smartphone's sensors (camera and microphone). For example, if the user inputs that they are "feeling stressed," the emotion engine will suggest recipes that contain ingredients with a relaxing effect. Or, if the user inputs that they are "feeling happy," the emotion engine will suggest special recipes suitable for celebrations. The input data is the user's emotional state, and the output is the adjusted recipe suggestions.

[0373] (Application example 1)

[0374] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0375] Conventional recipe suggestion systems could suggest recipes based on the contents of the user's refrigerator and preferences, but had the problem of not being able to make suggestions that took the user's emotional state into account. Even if there was a function to create a shopping list based on the suggested recipes and send it to an online supermarket, it was difficult to increase user satisfaction because it could not flexibly respond to the user's emotions. Furthermore, there was a lack of functionality to provide a path for users to make direct purchases, which made it less convenient.

[0376] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0377] In this invention, the server includes a means for grasping the contents of the user's refrigerator and each user's preferences, a means for proposing recipes based on the grasped information, a means for creating a shopping list according to the proposed recipes, a means for recognizing the user's emotions and adjusting the recipe suggestions according to the recognized emotions, and a means for transmitting the created shopping list to an online supermarket and providing a path for the user to make direct purchases. This makes it possible to propose recipes and create shopping lists according to the user's emotional state, thereby increasing user satisfaction and convenience.

[0378] "Means for understanding the contents of the user's refrigerator" refers to a means for recording and managing the types and quantities of food and beverages in the user's refrigerator.

[0379] "Means for understanding individual preferences" refers to means for understanding and recording a user's food preferences and tastes based on the user's past selection history and information directly entered by the user.

[0380] The "means for suggesting recipes" refers to a means for presenting appropriate recipes to the user based on the contents of the refrigerator and the user's preferences.

[0381] A "means for creating a shopping list" is a means for listing ingredients and seasonings needed for a suggested recipe and for the user to clearly indicate the items they should purchase.

[0382] A "means for recognizing a user's emotions" is a means for analyzing emotions from a user's input and actions and understanding their emotional state.

[0383] A "means for adjusting recipe suggestions according to emotions" is a means for changing the content and suggestion method of a recipe based on the recognized emotional state of the user, and presenting recipes that are appropriate to the user's emotions.

[0384] The "means for sending to the online supermarket" refers to a means for sending the created shopping list to an online supermarket so that the user can directly carry out the purchase procedure based on the list.

[0385] "Means for providing a path to direct purchase" refers to means for providing an interface such as a link or button that enables a user to make a purchase directly online based on the shopping list they created.

[0386] The system for implementing this invention will grasp the contents of the user's refrigerator and each individual's preferences, suggest recipes based on that, and create a shopping list. It will also have the function of recognizing the user's emotions and adjusting the recipe suggestions according to those emotions. Furthermore, it will send the created shopping list to an online supermarket, providing a path for the user to make direct purchases.

[0387] Hardware and Software Used

[0388] Hardware: Smartphone

[0389] software:

[0390] EmotionEngine: An engine for recognizing user emotions

[0391] RecipeEngine: An engine that suggests recipes based on what's in the fridge and the user's preferences

[0392] ShoppingListGenerator: An engine that generates shopping lists based on suggested recipes

[0393] requests: An HTTP request library for sending shopping lists to online supermarkets.

[0394] System processing overview

[0395] 1. Refrigerator Content Management: The user inputs the contents of the refrigerator through a smartphone application, which allows the system to track the types and quantities of food and beverages in the refrigerator.

[0396] 2. Understanding Preferences: Recording the user’s food preferences and tastes based on the user’s past ordering history and / or directly entered information.

[0397] 3. Emotion Recognition: Analyzes emotions from user input and behavior to understand the emotional state. For example, if a user inputs "I'm tired today," EmotionEngine will recognize "fatigue."

[0398] 4. Recipe Suggestion: Based on the recognized emotions, the contents of the refrigerator, and the user's preferences, RecipeEngine will suggest appropriate recipes. For example, if a user feels tired, it will suggest recipes that have a relaxing effect.

[0399] 5. Shopping list generation: The ingredients and seasonings required for the proposed recipes are listed, and the ShoppingListGenerator generates a shopping list.

[0400] 6. Online supermarket integration: The generated shopping list is sent to an online supermarket, allowing users to directly complete the purchase process based on the list.

[0401] Examples

[0402] When a user types "I'm tired today" into a smartphone application, the Emotion Engine recognizes the "fatigue" and suggests recipes that have a relaxing effect (e.g. herbal tea or light meals). Based on the suggested recipe, a shopping list containing the necessary ingredients is generated and sent to the online supermarket. The user can then directly check out the product based on the list.

[0403] Examples of prompt statements

[0404] When a user types "I'm tired today," the emotion engine should recognize this and suggest recipes that will help you relax. Then, generate a shopping list based on the suggested recipes and send it to the online supermarket.

[0405] In this way, recipe suggestions and shopping lists can be created according to the user's emotional state, increasing user satisfaction and convenience.

[0406] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0407] Step 1:

[0408] The user inputs the contents of the refrigerator through a smartphone application.

[0409] Input: Type and quantity of food and drinks in the refrigerator

[0410] Output: Refrigerator contents data

[0411] Specific operation: The user enters the ingredients in the refrigerator (e.g., tomatoes, cheese, lettuce) and their quantities in the application's input form, and presses the submit button. The server receives this data and stores it in the database.

[0412] Step 2:

[0413] Record user preferences based on past order history or directly entered information.

[0414] Input: Past order history, direct input of user preferences

[0415] Output: User preference data

[0416] Specific operation: The user enters preferences such as "I like Italian food" or "I'm vegetarian" in the app's settings screen. The server receives this information and stores it in a database.

[0417] Step 3:

[0418] The user inputs the emotion and the server recognizes the emotion.

[0419] Input: Input about the user's emotions (e.g. "I'm tired today")

[0420] Output: Recognized emotion data (e.g. "fatigue")

[0421] Specific operation: The user enters "I'm tired today" into the emotion input screen of the app and presses the send button. The server receives this input and recognizes it as "fatigue" using the Emotion Engine.

[0422] Step 4:

[0423] The server suggests recipes based on what's in the fridge, the user's preferences, and perceived emotions.

[0424] Input: fridge contents data, user preference data, recognized emotion data

[0425] Output: Proposed recipe data

[0426] How it works: The server uses the RecipeEngine to suggest relaxing recipes (e.g. herbal tea or snacks) based on the contents of the fridge (e.g. tomatoes, cheese, lettuce), the user's preferences (e.g. Italian, vegetarian), and perceived emotions (e.g. fatigue).

[0427] Step 5:

[0428] The server generates a shopping list based on the suggested recipes.

[0429] Input: Proposed recipe data

[0430] Output: Shopping list data

[0431] Specific behavior: The server uses a ShoppingListGenerator to generate a shopping list by listing the ingredients and condiments needed for the suggested recipes (e.g. herbs for herbal tea, bread for snacks).

[0432] Step 6:

[0433] The server transmits the generated shopping list to the online supermarket, providing a path for the user to make direct purchases.

[0434] Input: Shopping list data

[0435] Output: Order data for online supermarkets, purchase link

[0436] Specific operation: The server uses the HTTP request library (requests) to send the generated shopping list to the online supermarket's API. In response, the online supermarket returns a purchase link that is displayed to the user. The user can click on this link to proceed with the purchase directly.

[0437] Example 2

[0438] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0439] Conventional recipe suggestion systems have limited recipe suggestions based on the contents of the user's refrigerator and preferences, and have issues with not being able to suggest meals based on the user's emotional state. In addition, there are also issues with creating shopping lists based on suggested recipes and the cumbersome purchasing process at online supermarkets.

[0440] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0441] In this invention, the server includes a means for grasping the contents of the user's refrigerator and each person's preferences, a means for sending a prompt sentence to the generative AI model based on the grasped information and generating a recipe, a means for creating a shopping list based on the generated recipe, a means for sending the created shopping list to the online supermarket, and a means for the user to purchase the ingredients they need at the online supermarket. This makes it possible to suggest recipes based on the contents of the user's refrigerator and preferences, and also to suggest meals according to the user's emotional state. In addition, the process from creating the shopping list to the purchase procedure at the online supermarket can be done in a consistent manner, improving user convenience.

[0442] A "user" is an individual who uses the system to input the contents of their refrigerator and their preferred cooking genres and receives recipe suggestions and shopping lists.

[0443] "What's in the refrigerator" refers to information about ingredients and foods currently stored in the user's refrigerator.

[0444] A "favorite cuisine genre" is a particular style or type of cuisine selected by a user (e.g., Japanese, Western, Chinese, etc.).

[0445] A "generative AI model" is an artificial intelligence algorithm that generates appropriate recipes based on user input information.

[0446] A "prompt" is a text-based instruction sent to a generative AI model to request recipe generation.

[0447] A "recipe" is a list of instructions and ingredients needed to make a particular dish.

[0448] A "shopping list" is a list of ingredients and seasonings needed based on the generated recipe.

[0449] An "online supermarket" is an e-commerce platform where you can purchase food and daily necessities online.

[0450] An "emotional state" is a user's current psychological or emotional state (e.g. feeling stressed, wanting to relax, etc.) that they input into the system.

[0451] This invention is a system that identifies the contents of a user's refrigerator and their preferred cooking genres, uses a generative AI model to suggest recipes based on that information, and also allows them to purchase the necessary ingredients from an online supermarket.

[0452] The server receives information about the contents of the refrigerator and the user's favorite food genres input through the terminal. For example, if the user inputs "eggs and bacon" and selects "Japanese food" as a preference, the server generates a prompt based on this information.

[0453] Examples of specific prompts are as follows:

[0454] I have eggs and bacon in the fridge. Can you suggest some Japanese recipes?

[0455] The server sends this prompt to a generative AI model (e.g., GPT-4). The generative AI model analyzes the prompt and generates an appropriate recipe. For example, a recipe for "Japanese-style scrambled eggs with eggs and bacon" is generated.

[0456] Next, the server creates a shopping list based on the recipe, listing the condiments and vegetables needed to accompany the meal. For example, the following shopping list is created:

[0457] soy sauce

[0458] Mirin

[0459] spinach

[0460] The server then sends this shopping list via the API of the online supermarket the user uses, allowing the user to purchase the ingredients they need all at once via the online supermarket's application or website.

[0461] Furthermore, if the user inputs "I'm feeling stressed" into the emotion engine, the server can send an additional prompt to the generative AI model, suggesting a relaxing drink (e.g., "chamomile tea").

[0462] Examples of specific prompts are as follows:

[0463] There are eggs and bacon in the fridge. Can you suggest some Japanese recipes? Also, the user is feeling stressed so can you suggest some drinks that will help them relax?

[0464] In this way, it is possible to suggest recipes based on the contents of the user's refrigerator and preferences, and even meal suggestions based on the user's emotional state. It also makes it possible to create a shopping list and complete the purchase process at the online supermarket in one go, improving user convenience.

[0465] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0466] Step 1:

[0467] The user inputs the contents of the refrigerator and their favorite cuisine genre.

[0468] The user uses an application on a device (smartphone or PC) to input information about ingredients in the refrigerator (e.g. eggs and bacon) and their preferred cuisine (e.g. Japanese food). The input data is sent to the server.

[0469] Input: contents of fridge (eggs, bacon), favorite food genre (Japanese food)

[0470] Output: User input data sent to the server

[0471] Step 2:

[0472] The server receives the input data and sends prompts to the generative AI model.

[0473] The server receives the ingredient information and cuisine genre sent by the user, then generates a prompt based on this information and sends it to the generative AI model.

[0474] Specific prompt examples:

[0475] I have eggs and bacon in the fridge. Can you suggest some Japanese recipes?

[0476] Input: User input data (eggs, bacon, Japanese food)

[0477] Output: The prompt sent to the generative AI model

[0478] Step 3:

[0479] The generative AI model generates a recipe and returns it to the server.

[0480] The generative AI model receives the prompt and generates an appropriate recipe, for example, "Japanese-style scrambled eggs with eggs and bacon." The generated recipe is returned to the server.

[0481] Input: Prompt statement

[0482] Output: Generated recipe (Japanese-style scrambled eggs with eggs and bacon)

[0483] Step 4:

[0484] The server creates a shopping list based on the recipe.

[0485] The server analyzes the recipe returned by the generative AI model and lists the necessary seasonings, side dishes, etc. This creates a shopping list.

[0486] A concrete shopping list example:

[0487] soy sauce

[0488] Mirin

[0489] spinach

[0490] Input: Generated recipe

[0491] Output: Shopping list

[0492] Step 5:

[0493] The server sends the shopping list to the online supermarket.

[0494] The server sends the created shopping list via the API of the online supermarket the user uses, allowing the user to easily purchase the ingredients they need from the online supermarket.

[0495] Input: Shopping list

[0496] Output: Shopping list sent to online supermarket

[0497] Step 6:

[0498] The user purchases the necessary ingredients from an online supermarket.

[0499] Users can check the shopping list sent from the server via the online supermarket's application or website and purchase the ingredients they need all at once.

[0500] Input: Shopping list sent to online supermarket

[0501] Output: Purchased ingredients

[0502] (Application example 2)

[0503] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0504] Conventional recipe suggestion systems can suggest recipes based on the contents of the user's refrigerator and preferences, but they have a problem in that they cannot suggest recipes that take into account the user's emotional state. In addition, there is no mechanism for easily purchasing the ingredients needed for the suggested recipes, which results in low user convenience.

[0505] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0506] In this invention, the server includes a means for grasping the contents of the user's refrigerator and each user's preferences, a means for suggesting recipes based on the grasped information, a means for creating a shopping list according to the suggested recipes, a means for grasping the user's emotional state, a means for suggesting drinks with a relaxing effect according to the emotional state, and a means for transmitting the created shopping list to an online supermarket and providing a path for the user to make direct purchases. This enables recipe suggestions that take the user's emotional state into account and easy purchase of the necessary ingredients.

[0507] "User" means an individual or organization that uses the System.

[0508] "Contents of the refrigerator" refers to the food and beverages stored in the user's refrigerator.

[0509] "Preferences" refers to the preferences or tastes a user has for particular ingredients or dishes.

[0510] A "recipe" is a list of instructions and ingredients for making a particular dish.

[0511] A "shopping list" is a list of ingredients, seasonings, etc. that a user plans to purchase.

[0512] An "emotional state" is the emotion or mood a user is feeling at a particular point in time.

[0513] A "relaxing drink" is a drink that has the effect of reducing stress and relaxing the user.

[0514] An "online supermarket" is an online store that sells food and daily necessities via the Internet.

[0515] A "path" is the procedure or flow that a user follows to perform a specific operation or action through a system.

[0516] A system for implementing this invention includes means for grasping the contents of a user's refrigerator and each user's preferences, means for suggesting recipes based on the grasped information, means for creating a shopping list based on the suggested recipes, means for grasping the user's emotional state, means for suggesting drinks that have a relaxing effect according to the emotional state, and means for transmitting the created shopping list to an online supermarket and providing a path for the user to make direct purchases.

[0517] Hardware and Software Configuration

[0518] Hardware:

[0519] Smartphone

[0520] server

[0521] software:

[0522] Python

[0523] The Requests library (for sending HTTP requests)

[0524] Database management system (e.g. MySQL)

[0525] Emotion recognition engine (e.g. IBM Watson(R))

[0526] Data processing and calculation

[0527] Understanding what's in your fridge and what you like:

[0528] Through a smartphone application, users input the ingredients they have in their fridge and their preferences, and this data is sent to a server and stored in a database.

[0529] Recipe suggestions:

[0530] The server uses a generative AI model to suggest appropriate recipes based on the stored ingredients and preference data. For example, if a user inputs "eggs" and "bacon" and selects "Japanese food" as their preference, the system will suggest a recipe for "Japanese-style scrambled eggs with eggs and bacon."

[0531] Emotional state recognition and relaxing drink suggestions:

[0532] Users input their current emotional state through the emotion recognition engine. For example, if they input that they are "stressed," the system will suggest "chamomile tea," which has a relaxing effect.

[0533] Create and send a shopping list:

[0534] Based on the suggested recipe, a shopping list is automatically generated, including the necessary condiments and accompanying vegetables, and this list is sent through the online supermarket's API for easy purchase by the user.

[0535] Examples

[0536] If a user inputs that they have "eggs" and "bacon" in the fridge, selects "Japanese food" as their preference, and inputs "feeling stressed" as their emotional state, the system will suggest a recipe for "Japanese-style scrambled eggs with eggs and bacon" and add "chamomile tea" to the shopping list.

[0537] Example prompt for a generative AI model:

[0538] If a user inputs that they have eggs and bacon in the fridge, selects Japanese food as their preference, and indicates that they are stressed as their emotional state, suggest suitable recipes, add the necessary condiments and vegetable garnishes to their shopping list, and also suggest drinks that will have a relaxing effect.

[0539] In this way, personalized meal suggestions and shopping lists can be generated according to the user's needs.

[0540] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0541] Step 1:

[0542] The user launches a smartphone application and enters the ingredients they have in their fridge and their personal preferences.

[0543] Input: Ingredients (e.g. eggs, bacon), Preferences (e.g. Japanese food)

[0544] Output: The input data is sent to the server.

[0545] Specific behavior: The user enters ingredients and preferences into the application's input form, presses the "Submit" button, and the data is sent to the server.

[0546] Step 2:

[0547] The server stores the received data in a database.

[0548] Input: User-submitted ingredient and preference data

[0549] Output: Ingredients and preferences stored in a database

[0550] Specific operation: The server analyzes the received data and stores it in the corresponding table in the database.

[0551] Step 3:

[0552] Based on the data stored on the server, the server uses a generative AI model to suggest appropriate recipes.

[0553] Input: Ingredients and preferences stored in a database

[0554] Output: Suggested recipe (e.g. Japanese-style scrambled eggs with bacon)

[0555] Specific operation: The server inputs the prompt sentence into the generative AI model, generates an appropriate recipe, and returns the generated recipe to the user.

[0556] Step 4:

[0557] The user inputs their current emotional state through an emotion recognition engine.

[0558] Input: Emotional state (e.g. feeling stressed)

[0559] Output: The input data is sent to the server.

[0560] Specific behavior: When a user enters their emotional state into the application's emotion input form and presses the "Submit" button, the data is sent to the server.

[0561] Step 5:

[0562] The server suggests relaxing drinks based on the emotional state received.

[0563] Input: Emotional state data submitted by the user

[0564] Output: Suggested drink (e.g. chamomile tea)

[0565] What it does: The server analyzes your emotional state and suggests relaxing drinks.

[0566] Step 6:

[0567] The server generates a shopping list based on the suggested recipe and drink, including the necessary condiments and vegetable side dishes.

[0568] Input: Suggested recipe and drink

[0569] Output: Shopping list (e.g. soy sauce, mirin, green onion, chamomile tea)

[0570] What happens: The server parses the proposed recipe and drink and lists the required ingredients.

[0571] Step 7:

[0572] The server sends the generated shopping list to the online supermarket's API, allowing the user to make purchases directly.

[0573] Input: Shopping list

[0574] Output: Order status to online supermarket

[0575] Specific behavior: The server sends the shopping list to the online supermarket's API endpoint and checks whether the order was processed successfully.

[0576] In this way, personalized meal suggestions and shopping lists can be generated according to the user's needs.

[0577] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0578] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0579] Another example of generative AI is Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0580] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0581] [Second embodiment]

[0582] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0583] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0584] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0585] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0586] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0587] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0588] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0589] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0590] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0591] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0592] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0593] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0594] "Example 1"

[0595] In one embodiment of the present invention, a user inputs the contents of their refrigerator into a smartphone application. The user's preferences are also understood from their past recipe selection history and from direct input. Based on this information, the system suggests recipes to the user. Based on the suggested recipes, the system automatically creates a shopping list. The created shopping list is sent to the online supermarket that the user uses, and the user can make purchases directly based on the list.

[0596] "Example 2"

[0597] As a specific example, if a user inputs that they have eggs and bacon in the fridge and selects "Japanese food" as their preference, the system will suggest a recipe for "Japanese-style scrambled eggs with eggs and bacon." The system will then add the seasonings and accompanying vegetables needed for that recipe to a shopping list, which will be sent to the online supermarket the user uses. The user can then use that list to purchase the ingredients they need all at once.

[0598] The process flow of each embodiment will be described below.

[0599] "Example 1"

[0600] Step 1: The user opens the application on their smartphone and enters the contents of their refrigerator.

[0601] Step 2: The user inputs their preferences (e.g. Japanese, Western, Chinese, etc.) into the application.

[0602] Step 3: The system suggests recipes based on what's in the user's fridge and their preferences.

[0603] Step 4: The system automatically creates a shopping list based on the suggested recipes.

[0604] Step 5: The system sends the shopping list created by the user to the online supermarket that the user uses.

[0605] Step 6: The user purchases ingredients from the online supermarket based on the shopping list.

[0606] "Example 2"

[0607] Step 1: The user tells the application that they have eggs and bacon in the fridge.

[0608] Step 2: The user enters "Japanese food" as a preference into the application.

[0609] Step 3: The system suggests a recipe for "Japanese-style scrambled eggs with eggs and bacon."

[0610] Step 4: The system adds the condiments and vegetables needed for the recipe to a shopping list.

[0611] Step 5: The system sends the shopping list created by the user to the online supermarket that the user uses.

[0612] Step 6: The user purchases ingredients from the online supermarket based on the shopping list.

[0613] Example 1

[0614] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0615] In conventional systems, the process of users managing the contents of their refrigerator, finding recipes that matched their tastes, and purchasing the necessary ingredients was cumbersome. In addition, users would create shopping lists without considering the ingredients they already had, which could result in unnecessary purchases. Furthermore, the purchasing process at online supermarkets was done manually, which was time-consuming and cumbersome.

[0616] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0617] In this invention, the server includes a means for a user to input the contents of the refrigerator, a means for grasping the user's preferences, a means for proposing recipes using a generative AI model based on the grasped information, a means for automatically creating a shopping list based on the proposed recipes, a means for transmitting the created shopping list to an online supermarket, and a means for providing a path for the user to directly purchase at the online supermarket. This enables the user to efficiently manage the contents of the refrigerator, easily find recipes that suit their preferences, and purchase the necessary ingredients without waste.

[0618] "Means for users to input the contents of their refrigerator" refers to an interface that allows users to input the ingredients in their refrigerator using a smartphone or other device.

[0619] "Means of understanding user preferences" refers to a function for collecting and storing a user's food preferences based on the user's past recipe selection history or direct input.

[0620] "Means of suggesting recipes using a generative AI model" refers to a function that uses a generative AI model to generate and suggest appropriate recipes based on data on the contents of the user's refrigerator and their preferences.

[0621] The "means for automatically generating a shopping list" is a function for automatically generating a list of ingredients needed based on a suggested recipe.

[0622] "Means for sending a shopping list to an online supermarket" is a function for sending the created shopping list to the online supermarket used by the user.

[0623] "Means of providing a path for users to make purchases directly from online supermarkets" refers to a function that enables users to purchase groceries directly on the online supermarket platform based on the shopping list submitted to them.

[0624] The present invention is a system that allows a user to manage the contents of a refrigerator, suggest recipes that suit the user's preferences, and purchase the necessary ingredients efficiently. A specific embodiment of this system will be described below.

[0625] System Configuration

[0626] Hardware and Software

[0627] User device: Mobile device such as smartphone or tablet, using applications that run on iOS and Android.

[0628] Server: Located on the cloud, it hosts the database and generative AI model. It uses MySQL for the database and OpenAI's GPT-4 for the generative AI model.

[0629] Entering and Saving Data

[0630] The user enters the contents of the refrigerator

[0631] A user opens the smartphone application and inputs the ingredients in the refrigerator, either manually, by voice, by scanning a barcode, etc. For example, a user inputs "chicken, carrots, onions."

[0632] Understand user preferences

[0633] The user's preferences are captured from past recipe selection history or direct input. For example, if a user has previously selected "curry" as their favorite dish, that information is also stored in the database.

[0634] Recipe Suggestions

[0635] Server suggests recipes

[0636] The server uses a generative AI model to suggest recipes based on the contents of the user's fridge and their preferences. Specifically, it sends the following prompt to the generative AI model:

[0637] Contents of user's fridge: Chicken, carrots, onions

[0638] User preference: Curry

[0639] Generate suggested recipes.

[0640] The generative AI model generates a recipe for "chicken curry" based on this prompt, and the server sends the recipe to the user's smartphone.

[0641] Create and send a shopping list

[0642] The server creates the shopping list.

[0643] When a user selects a suggested recipe, the server automatically creates a shopping list of ingredients needed for that recipe, for example, "chicken, carrots, onions, curry roux, and any other necessary seasonings."

[0644] The server sends the shopping list to the online supermarket.

[0645] The server then sends the created shopping list to the online supermarket the user uses, using an API to link with the online supermarket's system.

[0646] Making a purchase

[0647] The user purchases from the online supermarket

[0648] The user opens the online supermarket's application or website, checks the shopping list sent from the server, and can then directly purchase the ingredients they need based on that list.

[0649] Examples

[0650] For example, if a user inputs that they have "chicken, carrots, and onions" in their fridge and has previously preferred "curry," the system will suggest a recipe for "chicken curry." Here is an example of a prompt to be input to the generative AI model:

[0651] Contents of user's fridge: Chicken, carrots, onions

[0652] User preference: Curry

[0653] Generate suggested recipes.

[0654] Based on this prompt, the generative AI model generates a recipe for "chicken curry" and suggests it to the user. Based on the suggested recipe, the system creates a shopping list of "chicken, carrots, onions, curry roux, and other necessary seasonings" and sends it to the online supermarket. The user can use this list to directly purchase the necessary ingredients.

[0655] In this way, users can efficiently manage the contents of their refrigerator, easily find recipes that suit their tastes, and buy the ingredients they need without waste.

[0656] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0657] Step 1:

[0658] The user enters the contents of the refrigerator

[0659] The user opens the smartphone application and inputs the ingredients in the refrigerator. Input methods include manual input, voice input, barcode scanning, etc. For example, the user inputs "chicken, carrots, onions." This input data is stored in a database on the cloud. The input data includes the names of ingredients and their quantities.

[0660] Step 2:

[0661] Understand user preferences

[0662] A user's preferences are understood from past recipe selection history or direct input. For example, if a user has previously selected "curry" as their favorite food, that information is also stored in the database. The server analyzes the user's selection history and extracts preference trends. The input data is the past recipe selection history or direct input of the user's preferences, and the output data is the user's preference trends.

[0663] Step 3:

[0664] Server suggests recipes

[0665] The server uses a generative AI model to suggest recipes based on the contents of the user's fridge and their preferences. Specifically, it sends the following prompt to the generative AI model:

[0666] Contents of user's fridge: Chicken, carrots, onions

[0667] User preference: Curry

[0668] Generate suggested recipes.

[0669] The generative AI model generates a recipe for "chicken curry" based on this prompt, and the server sends the recipe to the user's smartphone. The input data is the contents of the refrigerator and the user's preferences, and the output data is the suggested recipe.

[0670] Step 4:

[0671] The server creates the shopping list.

[0672] When a user selects a suggested recipe, the server automatically creates a shopping list of ingredients needed based on the recipe. For example, it generates a list of "chicken, carrots, onions, curry roux, and other necessary seasonings." The server also provides a function to remove ingredients from the list that the user already has. The input data is the selected recipe and the contents of the refrigerator, and the output data is the shopping list.

[0673] Step 5:

[0674] The server sends the shopping list to the online supermarket.

[0675] The server sends the created shopping list to the online supermarket that the user uses. An API is used to send the list, linking with the online supermarket's system. The input data is the shopping list and the online supermarket's API information, and the output data is the shopping list sent to the online supermarket.

[0676] Step 6:

[0677] The user purchases from the online supermarket

[0678] The user opens the online supermarket application or website and checks the shopping list sent from the server. The user can directly purchase the ingredients he needs based on the list. The input data is the shopping list sent to the online supermarket, and the output data is the purchased ingredients.

[0679] (Application example 1)

[0680] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0681] Conventional refrigerator management systems had the problem that users had to manually manage the contents of their refrigerators, search for recipes themselves, and list the ingredients they needed to purchase, which was a lot of work. In addition, it was difficult to suggest recipes that matched the user's preferences, and creating shopping lists and purchasing procedures were cumbersome. This reduced user convenience and made efficient ingredient management difficult.

[0682] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0683] In this invention, the server includes a means for grasping the contents of the user's refrigerator and each user's preferences, a means for proposing recipes based on the grasped information, a means for creating a shopping list according to the proposed recipes, and a means for transmitting the created shopping list to an electronic commerce platform and providing a path for the user to make direct purchases. This enables the user to efficiently manage the contents of the refrigerator, receive recipe suggestions according to their preferences, and easily purchase the necessary ingredients.

[0684] A "user" is an individual who uses the system to input the contents of their refrigerator and their preferences and receive recipe suggestions and shopping lists.

[0685] "What's in the refrigerator" refers to information about the food and beverages currently stored in the user's refrigerator.

[0686] "Preferences" are the ingredients and cooking preferences that a user indicates based on their past recipe selection history or direct input.

[0687] A "recipe" is a list of instructions and ingredients needed to make a particular dish.

[0688] A "shopping list" is a list of ingredients and products to purchase that is created based on a suggested recipe.

[0689] An "e-commerce platform" is an online marketplace for purchasing goods and services via the Internet.

[0690] A "guideline" is the sequence of steps or operations that allow a user to directly purchase items based on their shopping list.

[0691] A "generative AI model" is an artificial intelligence algorithm used to suggest optimal recipes based on what's in a user's fridge and their preferences.

[0692] A "prompt sentence" is an instruction sentence for suggesting recipes or creating a shopping list that is generated by a generative AI model based on information input by the user.

[0693] A system for implementing the present invention includes means for identifying the contents of a user's refrigerator and each user's preferences, means for suggesting recipes based on the identified information, means for creating a shopping list based on the suggested recipes, and means for transmitting the created shopping list to an e-commerce platform and providing a path for the user to make direct purchases.

[0694] System Programming

[0695] The server provides an interface for users to input the contents of their refrigerator using their smartphones. The user's preferences are understood from their past recipe selection history and direct input. This information is sent to the server and stored in a database.

[0696] The server uses a generative AI model to suggest optimal recipes based on the contents of the user's refrigerator and their preferences. The generative AI model generates prompts based on the user's input information and suggests recipes.

[0697] Based on the suggested recipes, the server automatically creates a shopping list, which is then sent to the e-commerce platform for users to purchase directly.

[0698] Hardware and Software Used

[0699] Hardware: Smartphones, servers

[0700] Software: Python, Requests library, generative AI models

[0701] Data processing and calculation

[0702] 1. User inputs the contents of the refrigerator: The user inputs the contents of the refrigerator using a smartphone application. This information is sent to the server and stored in a database.

[0703] 2. Understanding user preferences: User preferences are understood from past recipe selection history and direct input. This information is also sent to the server and stored in the database.

[0704] 3. Recipe Suggestion: The server uses a generative AI model to suggest optimal recipes based on the contents of the user's refrigerator and their preferences. The generative AI model generates prompts based on the user's input information and makes recipe suggestions.

[0705] 4. Create shopping list: Based on the suggested recipes, the server automatically creates a shopping list.

[0706] 5. Send shopping list: Once created, the shopping list is sent to the e-commerce platform to allow the user to make purchases directly.

[0707] Examples

[0708] Suppose a user inputs that they have “Ingredient A” and “Ingredient B” in their fridge, and that they prefer “vegetarian” and “non-spicy” food. If they have previously selected “Recipe A” and “Recipe B,” the system will suggest “Recipe C” and “Recipe D.” It will then generate a shopping list that includes “Ingredient 1” and “Ingredient 2” and send it to the e-commerce platform.

[0709] Examples of prompt statements

[0710] User ID: 12345

[0711] Contents of the fridge: ["ingredient A", "ingredient B"]

[0712] User Preferences: {"Vegetarian": True, "Spicy": False}

[0713] Past recipe selection history: ["Recipe A", "Recipe B"]

[0714] Suggested recipes: ["Recipe C", "Recipe D"]

[0715] Generated shopping list: ["ingredient 1", "ingredient 2"]

[0716] Sending status to online supermarket: 200

[0717] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0718] Step 1:

[0719] The user inputs the contents of the refrigerator using a smartphone application.

[0720] Input: Information about ingredients in the user's refrigerator (e.g., ingredient A, ingredient B)

[0721] Data processing: The entered ingredient information is converted into JSON format and sent to the server.

[0722] Output: The contents of the fridge sent to the server.

[0723] Step 2:

[0724] The server knows the user's preferences.

[0725] Input: User's past recipe selection history and direct preference input (e.g. vegetarian, not spicy)

[0726] Data processing: Store user preference information in a database.

[0727] Output: User preferences stored in a database

[0728] Step 3:

[0729] The server suggests recipes using a generative AI model.

[0730] Input: Fridge contents data, user preferences data

[0731] Data processing: A generative AI model uses this data to generate prompts and suggest optimal recipes.

[0732] Output: Suggested recipes (e.g. Recipe C, Recipe D)

[0733] Step 4:

[0734] The server creates a shopping list based on the suggested recipes.

[0735] Input: Suggested recipe data

[0736] Data processing: List ingredients needed for recipes and generate shopping lists.

[0737] Output: The created shopping list (e.g. ingredient 1, ingredient 2)

[0738] Step 5:

[0739] The server transmits the created shopping list to the e-commerce platform.

[0740] Input: The shopping list data created

[0741] Data processing: Formatting the shopping list to send to the e-commerce platform's API.

[0742] Output: Status of submission to e-commerce platform (e.g. 200 OK)

[0743] Step 6:

[0744] A user purchases products based on a shopping list through an e-commerce platform.

[0745] Input: Shopping list submitted to e-commerce platform

[0746] Data processing: The e-commerce platform searches for products based on the shopping list and completes the purchase.

[0747] Output: A confirmation message for the purchase (e.g. Purchase Completed)

[0748] Example 2

[0749] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0750] Conventional recipe suggestion systems would only suggest appropriate recipes when users input the ingredients they had in their fridge and their preferred cooking genre, but lacked the functionality to automatically list the necessary seasonings and accompaniments and send them to the online supermarket. As a result, users had to manually create a shopping list based on the suggested recipes and then go through the purchasing process separately at the online supermarket. This process was time-consuming and inefficient.

[0751] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0752] In this invention, the server includes a means for grasping the contents of a user's refrigerator and each user's preferences, a means for proposing a recipe using a generative AI model based on the grasped information, a means for creating a shopping list including necessary seasonings and accompaniment ingredients based on the proposed recipe, a means for transmitting the created shopping list to an online supermarket, and a means for providing a path for the user to purchase the necessary ingredients in one go through the online supermarket. As a result, the user is automatically suggested appropriate recipes based on the contents of the refrigerator and their preferences, and is able to efficiently purchase the necessary ingredients.

[0753] "User" refers to an individual who uses the system to input the contents of their refrigerator and their preferred cuisine genres.

[0754] "What's in the refrigerator" refers to the food items currently stored in the user's refrigerator.

[0755] "Favorite Cuisine Genre" refers to a particular style or type of cuisine selected by a User.

[0756] A "generative AI model" refers to an artificial intelligence algorithm that generates an appropriate recipe based on input information.

[0757] A "prompt sentence" refers to an input sentence used to request a generative AI model to generate a recipe.

[0758] A "recipe" is a list of instructions or ingredients for making a particular dish.

[0759] "Condiments" refer to ingredients used to adjust the flavor of food.

[0760] "Accompaniment ingredients" refers to additional ingredients served with the main dish.

[0761] A "shopping list" refers to a list of ingredients needed based on a suggested recipe.

[0762] An "online supermarket" refers to a website or application that provides the service of purchasing food and daily necessities online.

[0763] "Path" refers to the steps and flow that users take to purchase the ingredients they need in one go through an online supermarket.

[0764] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0765] This invention is a system that, when a user inputs the ingredients in the refrigerator and the type of cooking they like, suggests appropriate recipes, automatically creates a shopping list including the necessary seasonings and garnishes, and transmits it to an online supermarket. A specific embodiment of this system is described below.

[0766] 1. System Overview

[0767] The system involves a series of processes: identifying the contents of a user's refrigerator and preferred cooking genres, suggesting recipes using a generative AI model, creating a shopping list based on the suggested recipes, and sending it to the online supermarket.

[0768] 2. Hardware and Software Used

[0769] Hardware: The device used by the user (smartphone, tablet, computer, etc.)

[0770] Software: Server, generative AI model (e.g. OpenAI's GPT-4), online supermarket API

[0771] 3. Program Processing

[0772] 1. Receiving User Input

[0773] The user inputs into the device the ingredients they have in their fridge (e.g. eggs and bacon) and their preferred cuisine type (e.g. Japanese food).

[0774] The terminal transmits the input information to the server.

[0775] 2. Recipe suggestions

[0776] The server analyzes the received information on ingredients and cuisine genre.

[0777] The server sends prompts to the generative AI model, which then generates an appropriate recipe.

[0778] The server sends the generated recipe (e.g. "Japanese-style scrambled eggs with eggs and bacon") back to the terminal.

[0779] 3. Generate a shopping list

[0780] The server analyzes the generated recipe and lists the necessary condiments and side dishes.

[0781] The server sends this shopping list to the API of the online supermarket the user is using.

[0782] 4. Purchase procedure

[0783] The user accesses the online supermarket's website through their device and checks the submitted shopping list.

[0784] Users purchase the ingredients they need in bulk.

[0785] 4. Specific Examples

[0786] The user types into the device, "I have eggs and bacon in the fridge," and selects "Japanese food" as their preferred cuisine genre.

[0787] The server uses a generative AI model to suggest a recipe for "Japanese-style scrambled eggs with eggs and bacon."

[0788] The server adds the necessary seasonings (e.g. soy sauce, mirin) and accompanying vegetables (e.g. spinach) to a shopping list and sends it to the online supermarket.

[0789] Users check their shopping list on the online supermarket website and purchase all the ingredients they need in one go.

[0790] 5. Examples of prompts

[0791] I have eggs and bacon in the fridge. Can you suggest some Japanese recipes?

[0792] The above is a specific embodiment for carrying out the present invention. This system automatically suggests suitable recipes to users based on the contents of their refrigerator and their preferences, enabling them to efficiently purchase the ingredients they need.

[0793] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0794] Program processing flow

[0795] Step 1: Receiving User Input

[0796] The user inputs into the device the ingredients they have in their fridge (e.g. eggs and bacon) and their preferred cuisine type (e.g. Japanese food).

[0797] Input: A user types "eggs and bacon" into a terminal application and selects "Japanese food" from a drop-down menu.

[0798] The terminal transmits the input information to the server.

[0799] Output: The data sent from the terminal to the server is ingredient information and cuisine genre information.

[0800] Specific actions: A user opens an application on their smartphone, types "eggs and bacon" into the text box, selects "Japanese food" from the drop-down menu, and presses the "Submit" button.

[0801] Step 2: Suggest a recipe

[0802] The server analyzes the received information on ingredients and cuisine genre.

[0803] Input: Ingredient information and cuisine genre information received by the server from the terminal.

[0804] The server sends prompts to the generative AI model, which then generates an appropriate recipe.

[0805] Data processing: The server sends a prompt to the generative AI model: "There are eggs and bacon in the fridge. Please suggest a Japanese recipe."

[0806] Output: Recipe information returned from the generative AI model.

[0807] The server sends the generated recipe (e.g. "Japanese-style scrambled eggs with eggs and bacon") back to the terminal.

[0808] Specific operation: The server sends a prompt to the generative AI model and displays the returned recipe on the user's device.

[0809] Step 3: Generate a shopping list

[0810] The server analyzes the generated recipe and lists the necessary condiments and side dishes.

[0811] Input: The generated recipe information.

[0812] Data processing: The server extracts the necessary seasonings (e.g. soy sauce, mirin) and accompanying vegetables (e.g. spinach) from the recipe.

[0813] Output: A list of ingredients needed.

[0814] The server sends this shopping list to the API of the online supermarket the user is using.

[0815] Specific behavior: The server sends a request to the online supermarket's API saying, "Please add these items to your shopping list."

[0816] Step 4: Checkout

[0817] The user accesses the online supermarket's website through their device and checks the submitted shopping list.

[0818] Input: The shopping list sent by the server.

[0819] Output: Shopping list displayed on the online supermarket website.

[0820] Users purchase the ingredients they need in bulk.

[0821] Specific behavior: The user opens the online supermarket app, goes to the "Shopping List" section, checks the list sent from the server, and presses the "Purchase" button.

[0822] The above is the specific processing flow of the program of this system.

[0823] (Application example 2)

[0824] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0825] Conventional recipe suggestion systems were insufficient in suggesting recipes that took into account the ingredients in the user's refrigerator and preferences, and had problems with creating a shopping list based on the suggested recipe and the complicated process of purchasing the necessary ingredients. In addition, there was no way for users to easily purchase additional ingredients required for the suggested recipe, which resulted in low convenience. There is a need to solve these problems and improve user convenience.

[0826] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0827] In this invention, the server includes a means for grasping the contents of a user's refrigerator and each user's preferences, a means for proposing recipes using a generative AI model based on the grasped information, a means for creating a shopping list according to the proposed recipes, and a means for transmitting the created shopping list to a food delivery service and providing a path for the user to directly purchase. This allows the user to receive recipe suggestions based on the ingredients in the refrigerator and their preferences, and further allows the user to easily purchase any additional ingredients that are required.

[0828] A "user" is an individual who uses the system to input the contents of their refrigerator and their preferences and receive recipe suggestions and shopping lists.

[0829] "Contents of the refrigerator" refers to ingredients and foods currently stored in the user's refrigerator.

[0830] "Preferences" refers to a user's personal preferences for a particular cuisine genre or seasoning.

[0831] A "generative AI model" is an artificial intelligence model that generates appropriate recipes based on user input information.

[0832] A "recipe" is a set of instructions that lists the steps and ingredients needed to make a particular dish.

[0833] A "shopping list" is a list of ingredients and seasonings needed based on a suggested recipe.

[0834] A "food delivery service" is a service that delivers ingredients and meals ordered online by users to their homes.

[0835] A "guideline" refers to the flow of procedures and operations that allow users to easily purchase the ingredients they need through the system.

[0836] A system for implementing this invention includes means for understanding the contents of a user's refrigerator and their individual preferences, means for suggesting recipes using a generative AI model based on the understood information, means for creating a shopping list based on the suggested recipes, and means for sending the created shopping list to a food delivery service and providing a path for the user to make direct purchases.

[0837] Hardware and Software Configuration

[0838] Hardware: Smartphones, servers

[0839] Software: Python, OpenAI API, HTTP request library (requests)

[0840] Data processing and calculation

[0841] 1. Receiving user input: The user uses their smartphone to input the ingredients they have in their fridge and their preferred cuisine, for example, "eggs, bacon" and "Japanese food."

[0842] 2. Using a generative AI model: The server uses the OpenAI API to generate a recipe based on the user's input. An example of a prompt is as follows:

[0843] Ingredients in the fridge: eggs, bacon

[0844] Favorite food genre: Japanese food

[0845] Suggested recipe:

[0846] 3. Creating a shopping list: The server extracts the additional ingredients required from the generated recipe and creates a shopping list. For example, "soy sauce, mirin, and green onion" are added.

[0847] 4. Sending an order to a food delivery service: The server sends the created shopping list to the food delivery service and provides a path for the user to purchase directly. This allows the user to easily purchase the ingredients they need.

[0848] Examples

[0849] If a user inputs that they have "eggs and bacon" in their fridge and selects "Japanese" as their preferred cuisine genre, the server will use the generative AI model to suggest a recipe for "Japanese-style scrambled eggs with eggs and bacon." It will then add "soy sauce, mirin, and green onions" as additional ingredients needed to a shopping list and send this list to a food delivery service. The user can easily purchase these ingredients and have them delivered via their smartphone.

[0850] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0851] Step 1:

[0852] The user uses a smartphone to input the ingredients in the refrigerator and the type of cuisine they prefer.

[0853] Input: Ingredients in the fridge (e.g., "eggs, bacon"), favorite cuisine (e.g., "Japanese food")

[0854] Output: User input data

[0855] Specific actions: A user opens the application on their smartphone, enters ingredients in the fridge into a text box, and selects their preferred cuisine from a drop-down menu.

[0856] Step 2:

[0857] The server receives the user input data and generates the recipe using a generative AI model.

[0858] Input: User-entered data (ingredients in the fridge, favorite cuisine genres)

[0859] Output: Suggested recipe

[0860] Specific operation: The server calls the OpenAI API and sends the following prompt sentence to the generative AI model.

[0861] Ingredients in the fridge: eggs, bacon

[0862] Favorite food genre: Japanese food

[0863] Suggested recipe:

[0864] The generative AI model generates a recipe based on this prompt and returns it to the server.

[0865] Step 3:

[0866] The server extracts any additional ingredients required from the generated recipe and creates a shopping list.

[0867] Input: Suggested recipe

[0868] Output: Shopping list

[0869] Specific behavior: The server parses the generated recipe text and extracts any additional ingredients needed (e.g., "soy sauce, mirin, green onion"), organizes these ingredients into a list, and creates a shopping list.

[0870] Step 4:

[0871] The server creates a shopping list and sends it to the food delivery service, providing a path for the user to make a purchase directly.

[0872] Input: Shopping list

[0873] Output: Order data for food delivery service

[0874] What happens: The server calls the food delivery service's API and sends the shopping list as order data, including the user's address information, to ensure that the order is processed successfully.

[0875] Step 5:

[0876] The user receives an order confirmation from the food delivery service via smartphone and purchases the necessary ingredients.

[0877] Input: Order confirmation from food delivery service

[0878] Output: Purchased ingredients

[0879] How it works: The user receives an order confirmation on their smartphone app, waits for the delivery of the ingredients they need, and then collects the purchased ingredients when the delivery arrives.

[0880] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0881] "Example 1"

[0882] The present invention is a system including a means for grasping the contents of a user's refrigerator and each user's preferences, a means for suggesting recipes based on the grasped information, a means for creating a shopping list based on the suggested recipes, and an emotion engine for recognizing the user's emotions. The emotion engine has a function for adjusting the recipe suggestions and the shopping list according to the user's emotions. For example, when the user is feeling stressed, the emotion engine suggests recipes that include ingredients that have a relaxing effect. Also, when the user is feeling happy, the emotion engine suggests special recipes suitable for celebrations. This makes it possible to suggest meals according to the user's emotional state.

[0883] "Example 2"

[0884] As a concrete example, consider the case where a user inputs to the application that they have eggs and bacon in the fridge and selects "Japanese food" as their preference. Furthermore, suppose the user inputs to the emotion engine that they "feel stressed." In this case, the system will suggest a recipe for "Japanese-style scrambled eggs with eggs and bacon" and create a shopping list that adds "chamomile tea," which has a relaxing effect. This makes it possible to suggest meals according to the user's emotional state.

[0885] The process flow of each embodiment will be described below.

[0886] "Example 1"

[0887] Step 1: The user enters the contents of the fridge and each person's preferences into the application.

[0888] Step 2: The system will understand the information you entered and suggest recipes based on it.

[0889] Step 3: The system creates a shopping list with suggested recipes.

[0890] Step 4: The user inputs their current emotional state into the emotion engine.

[0891] Step 5: The emotion engine adapts recipe suggestions and shopping lists based on the user's emotions.

[0892] "Example 2"

[0893] Step 1: The user tells the application what eggs and bacon they have in the fridge, and that their preference is "Japanese food."

[0894] Step 2: The user types "I'm stressed" into the emotion engine.

[0895] Step 3: The system understands the input information and emotional state and suggests a recipe for "Japanese-style scrambled eggs with eggs and bacon."

[0896] Step 4: The system creates a shopping list to which it adds the relaxing "chamomile tea."

[0897] Example 1

[0898] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0899] Conventional recipe suggestion systems have problems in that it is difficult to suggest recipes that fully take into account the contents of the user's refrigerator and preferences, and they are unable to suggest recipes or adjust shopping lists according to the user's emotional state. Furthermore, creating a shopping list based on the suggested recipes and sending it to the online supermarket is done manually, which is time-consuming for users.

[0900] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0901] In this invention, the server includes means for inputting the contents of the user's refrigerator, means for grasping the user's preferences, means for suggesting recipes based on the grasped information, means for automatically creating a shopping list according to the suggested recipes, means for transmitting the created shopping list to the online supermarket, and means for recognizing the user's emotional state and adjusting the recipe suggestions and shopping list accordingly. This makes it possible to automatically create optimal recipe suggestions and shopping lists according to the contents of the user's refrigerator, preferences, and emotional state, and to transmit them to the online supermarket.

[0902] "Means for inputting the contents of the user's refrigerator" refers to an interface that allows a user to input the ingredients in their refrigerator via a smartphone or other device.

[0903] The "means of understanding user preferences" refers to a system that stores in a database the recipes the user has previously chosen, as well as the ingredients and types of dishes that the user has registered as their preferences, in order to understand the user's tastes.

[0904] The "means of suggesting recipes based on the captured information" refers to an algorithm that searches for and suggests appropriate recipes based on the contents of the user's refrigerator and their preferences.

[0905] The "means for automatically creating a shopping list according to a proposed recipe" is a system that lists ingredients needed for a proposed recipe and automatically generates a shopping list.

[0906] "Means for sending the created shopping list to the online supermarket" refers to a system that sends the created shopping list via the API of the online supermarket used by the user, enabling the user to make purchases directly.

[0907] "Means for recognizing the user's emotional state and adjusting recipe suggestions and shopping lists accordingly" refers to a system that recognizes the user's emotional state through sensors and input data, and suggests and adjusts recipes and shopping lists according to that state.

[0908] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0909] This invention is a system that proposes optimal recipes based on the contents of a user's refrigerator, preferences, and emotional state, and automatically creates and transmits a shopping list. A specific embodiment of this system will be described below.

[0910] Input the contents of the user's refrigerator

[0911] Users use a smartphone application to input the contents of their refrigerator. The application runs on iOS and Android platforms. Users enter ingredients such as "chicken, cabbage, and carrots" by text into the application's input form, or scan the barcodes of the ingredients using the barcode scanner function. The input data is sent from the device to the server and stored in a database.

[0912] Understanding user preferences

[0913] Users input recipes they have previously chosen within the application, as well as ingredients and types of cuisine they have registered as preferences. For example, if a user inputs that they "like Japanese food," that information is sent from the device to the server and stored in the user preference database. The server uses this data to determine the user's preferences.

[0914] Recipe Suggestions

[0915] The server suggests appropriate recipes based on the contents of the refrigerator and the user's preferences, using an algorithm that references the recipe database and the user's preference database. For example, if a user has "chicken, cabbage, and carrots," the server will suggest "stir-fried chicken and cabbage" or "teriyaki chicken."

[0916] Automatic shopping list creation

[0917] The server automatically creates a shopping list based on the proposed recipe. Specifically, the ingredients required for the recipe are added to the list. For example, if a recipe for "stir-fried chicken with vegetables" is proposed, the necessary ingredients (chicken, cabbage, carrots, seasonings, etc.) are added to the list. This list is sent to the user's device and displayed within the application.

[0918] Send to online supermarket

[0919] When a user checks the shopping list and wishes to purchase from an online supermarket, a request is sent from the terminal to the server. The server sends the shopping list through the API of the online supermarket the user is using. For example, the list is sent to a service such as an online shopping platform. The user can then make a purchase directly based on the list.

[0920] Emotion engine adjustment

[0921] The server recognizes the user's emotional state using the emotion engine. Specifically, the user inputs their emotional state within the application, or detects emotions through the smartphone's sensors (camera and microphone). For example, if the user inputs that they are "feeling stressed," the emotion engine will suggest recipes that contain ingredients that have a relaxing effect. Also, if the user inputs that they are "feeling happy," the emotion engine will suggest special recipes suitable for celebrations.

[0922] Examples of concrete examples and prompts

[0923] If a user inputs that they have "chicken, cabbage, and carrots" in their fridge and has previously registered that they like "Japanese food," the server will suggest "chicken and cabbage stir fry" or "teriyaki chicken." If the emotion engine recognizes that the user is stressed, it will also suggest "relaxing herbal tea."

[0924] Example prompt:

[0925] "I have some chicken, cabbage, and carrots in the fridge. I like Japanese food. I'm feeling stressed. Can you recommend some recipes?"

[0926] In this way, based on the contents of the user's fridge, preferences, and emotional state, the server suggests optimal recipes and shopping lists.

[0927] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0928] Step 1:

[0929] The user launches the smartphone application and inputs the contents of the refrigerator. Input methods include text input and barcode scanner functions. When the user inputs "chicken, cabbage, carrots," the data is sent from the device to the server. The server stores the received data in a database. The input data is a list of ingredients, and the output is the ingredient information stored in the database.

[0930] Step 2:

[0931] Within the application, the user inputs previously selected recipes, favorite ingredients, and types of cuisine. For example, if the user inputs "I like Japanese food," that information is sent from the device to the server and stored in a user preference database. The server uses this data to understand the user's preferences. The input data is the user's preference information, and the output is the preference information stored in the database.

[0932] Step 3:

[0933] The server suggests appropriate recipes based on the contents of the refrigerator and preference data entered by the user. Specifically, the server searches the recipe database and extracts recipes that match the contents of the user's refrigerator. For example, if the user has "chicken, cabbage, and carrots," the server will suggest "stir-fried chicken and cabbage" or "teriyaki chicken." The input data are the contents of the refrigerator and preference information, and the output is the suggested recipe.

[0934] Step 4:

[0935] The server automatically creates a shopping list based on the proposed recipe. Specifically, the ingredients required for the recipe are added to the list. For example, if a recipe for "stir-fried chicken with vegetables" is proposed, the necessary ingredients (chicken, cabbage, carrots, seasonings, etc.) are added to the list. This list is sent to the user's device and displayed within the application. The input data is the proposed recipe, and the output is the shopping list.

[0936] Step 5:

[0937] If the user checks the shopping list and wishes to make a purchase at the online supermarket, a request is sent from the terminal to the server. The server sends the shopping list through the API of the online supermarket the user is using. For example, the list is sent to a service such as an online shopping platform. The user can then make a purchase directly based on the list. The input data is the shopping list, and the output is the result of sending it to the online supermarket.

[0938] Step 6:

[0939] The server recognizes the user's emotional state using the emotion engine. Specifically, the user inputs their emotional state within the application or detects emotions through the smartphone's sensors (camera and microphone). For example, if the user inputs that they are "feeling stressed," the emotion engine will suggest recipes that contain ingredients with a relaxing effect. Or, if the user inputs that they are "feeling happy," the emotion engine will suggest special recipes suitable for celebrations. The input data is the user's emotional state, and the output is the adjusted recipe suggestions.

[0940] (Application example 1)

[0941] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0942] Conventional recipe suggestion systems could suggest recipes based on the contents of the user's refrigerator and preferences, but had the problem of not being able to make suggestions that took the user's emotional state into account. Even if there was a function to create a shopping list based on the suggested recipes and send it to an online supermarket, it was difficult to increase user satisfaction because it could not flexibly respond to the user's emotions. Furthermore, there was a lack of functionality to provide a path for users to make direct purchases, which made it less convenient.

[0943] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0944] In this invention, the server includes a means for grasping the contents of the user's refrigerator and each user's preferences, a means for proposing recipes based on the grasped information, a means for creating a shopping list according to the proposed recipes, a means for recognizing the user's emotions and adjusting the recipe suggestions according to the recognized emotions, and a means for transmitting the created shopping list to an online supermarket and providing a path for the user to make direct purchases. This makes it possible to propose recipes and create shopping lists according to the user's emotional state, thereby increasing user satisfaction and convenience.

[0945] "Means for understanding the contents of the user's refrigerator" refers to a means for recording and managing the types and quantities of food and beverages in the user's refrigerator.

[0946] "Means for understanding individual preferences" refers to means for understanding and recording a user's food preferences and tastes based on the user's past selection history and information directly entered by the user.

[0947] The "means for suggesting recipes" refers to a means for presenting appropriate recipes to the user based on the contents of the refrigerator and the user's preferences.

[0948] A "means for creating a shopping list" is a means for listing ingredients and seasonings needed for a suggested recipe and for the user to clearly indicate the items they should purchase.

[0949] A "means for recognizing a user's emotions" is a means for analyzing emotions from a user's input and actions and understanding their emotional state.

[0950] A "means for adjusting recipe suggestions according to emotions" is a means for changing the content and suggestion method of a recipe based on the recognized emotional state of the user, and presenting recipes that are appropriate to the user's emotions.

[0951] The "means for sending to the online supermarket" refers to a means for sending the created shopping list to an online supermarket so that the user can directly carry out the purchase procedure based on the list.

[0952] "Means for providing a path to direct purchase" refers to means for providing an interface such as a link or button that enables a user to make a purchase directly online based on the shopping list they created.

[0953] The system for implementing this invention will grasp the contents of the user's refrigerator and each individual's preferences, suggest recipes based on that, and create a shopping list. It will also have the function of recognizing the user's emotions and adjusting the recipe suggestions according to those emotions. Furthermore, it will send the created shopping list to an online supermarket, providing a path for the user to make direct purchases.

[0954] Hardware and Software Used

[0955] Hardware: Smartphone

[0956] software:

[0957] EmotionEngine: An engine for recognizing user emotions

[0958] RecipeEngine: An engine that suggests recipes based on what's in the fridge and the user's preferences

[0959] ShoppingListGenerator: An engine that generates shopping lists based on suggested recipes

[0960] requests: An HTTP request library for sending shopping lists to online supermarkets.

[0961] System processing overview

[0962] 1. Refrigerator Content Management: The user inputs the contents of the refrigerator through a smartphone application, which allows the system to track the types and quantities of food and beverages in the refrigerator.

[0963] 2. Understanding Preferences: Recording the user’s food preferences and tastes based on the user’s past ordering history and / or directly entered information.

[0964] 3. Emotion Recognition: Analyzes emotions from user input and behavior to understand the emotional state. For example, if a user inputs "I'm tired today," EmotionEngine will recognize "fatigue."

[0965] 4. Recipe Suggestion: Based on the recognized emotions, the contents of the refrigerator, and the user's preferences, RecipeEngine will suggest appropriate recipes. For example, if a user feels tired, it will suggest recipes that have a relaxing effect.

[0966] 5. Shopping list generation: The ingredients and seasonings required for the proposed recipes are listed, and the ShoppingListGenerator generates a shopping list.

[0967] 6. Online supermarket integration: The generated shopping list is sent to an online supermarket, allowing users to directly complete the purchase process based on the list.

[0968] Examples

[0969] When a user types "I'm tired today" into a smartphone application, the Emotion Engine recognizes the "fatigue" and suggests recipes that have a relaxing effect (e.g. herbal tea or light meals). Based on the suggested recipe, a shopping list containing the necessary ingredients is generated and sent to the online supermarket. The user can then directly check out the product based on the list.

[0970] Examples of prompt statements

[0971] When a user types "I'm tired today," the emotion engine should recognize this and suggest recipes that will help you relax. Then, generate a shopping list based on the suggested recipes and send it to the online supermarket.

[0972] In this way, recipe suggestions and shopping lists can be created according to the user's emotional state, increasing user satisfaction and convenience.

[0973] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0974] Step 1:

[0975] The user inputs the contents of the refrigerator through a smartphone application.

[0976] Input: Type and quantity of food and drinks in the refrigerator

[0977] Output: Refrigerator contents data

[0978] Specific operation: The user enters the ingredients in the refrigerator (e.g., tomatoes, cheese, lettuce) and their quantities in the application's input form, and presses the submit button. The server receives this data and stores it in the database.

[0979] Step 2:

[0980] Record user preferences based on past order history or directly entered information.

[0981] Input: Past order history, direct input of user preferences

[0982] Output: User preference data

[0983] Specific operation: The user enters preferences such as "I like Italian food" or "I'm vegetarian" in the app's settings screen. The server receives this information and stores it in a database.

[0984] Step 3:

[0985] The user inputs the emotion and the server recognizes the emotion.

[0986] Input: Input about the user's emotions (e.g. "I'm tired today")

[0987] Output: Recognized emotion data (e.g. "fatigue")

[0988] Specific operation: The user enters "I'm tired today" into the emotion input screen of the app and presses the send button. The server receives this input and recognizes it as "fatigue" using the Emotion Engine.

[0989] Step 4:

[0990] The server suggests recipes based on what's in the fridge, the user's preferences, and perceived emotions.

[0991] Input: fridge contents data, user preference data, recognized emotion data

[0992] Output: Proposed recipe data

[0993] How it works: The server uses the RecipeEngine to suggest relaxing recipes (e.g. herbal tea or snacks) based on the contents of the fridge (e.g. tomatoes, cheese, lettuce), the user's preferences (e.g. Italian, vegetarian), and perceived emotions (e.g. fatigue).

[0994] Step 5:

[0995] The server generates a shopping list based on the suggested recipes.

[0996] Input: Proposed recipe data

[0997] Output: Shopping list data

[0998] Specific behavior: The server uses a ShoppingListGenerator to generate a shopping list by listing the ingredients and condiments needed for the suggested recipes (e.g. herbs for herbal tea, bread for snacks).

[0999] Step 6:

[1000] The server transmits the generated shopping list to the online supermarket, providing a path for the user to make direct purchases.

[1001] Input: Shopping list data

[1002] Output: Order data for online supermarkets, purchase link

[1003] Specific operation: The server uses the HTTP request library (requests) to send the generated shopping list to the online supermarket's API. In response, the online supermarket returns a purchase link that is displayed to the user. The user can click on this link to proceed with the purchase directly.

[1004] Example 2

[1005] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[1006] Conventional recipe suggestion systems have limited recipe suggestions based on the contents of the user's refrigerator and preferences, and have issues with not being able to suggest meals based on the user's emotional state. In addition, there are also issues with creating shopping lists based on suggested recipes and the cumbersome purchasing process at online supermarkets.

[1007] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1008] In this invention, the server includes a means for grasping the contents of the user's refrigerator and each person's preferences, a means for sending a prompt sentence to the generative AI model based on the grasped information and generating a recipe, a means for creating a shopping list based on the generated recipe, a means for sending the created shopping list to the online supermarket, and a means for the user to purchase the ingredients they need at the online supermarket. This makes it possible to suggest recipes based on the contents of the user's refrigerator and preferences, and also to suggest meals according to the user's emotional state. In addition, the process from creating the shopping list to the purchase procedure at the online supermarket can be done in a consistent manner, improving user convenience.

[1009] A "user" is an individual who uses the system to input the contents of their refrigerator and their preferred cooking genres and receives recipe suggestions and shopping lists.

[1010] "What's in the refrigerator" refers to information about ingredients and foods currently stored in the user's refrigerator.

[1011] A "favorite cuisine genre" is a particular style or type of cuisine selected by a user (e.g., Japanese, Western, Chinese, etc.).

[1012] A "generative AI model" is an artificial intelligence algorithm that generates appropriate recipes based on user input information.

[1013] A "prompt" is a text-based instruction sent to a generative AI model to request recipe generation.

[1014] A "recipe" is a list of instructions and ingredients needed to make a particular dish.

[1015] A "shopping list" is a list of ingredients and seasonings needed based on the generated recipe.

[1016] An "online supermarket" is an e-commerce platform where you can purchase food and daily necessities online.

[1017] An "emotional state" is a user's current psychological or emotional state (e.g. feeling stressed, wanting to relax, etc.) that they input into the system.

[1018] This invention is a system that identifies the contents of a user's refrigerator and their preferred cooking genres, uses a generative AI model to suggest recipes based on that information, and also allows them to purchase the necessary ingredients from an online supermarket.

[1019] The server receives information about the contents of the refrigerator and the user's favorite food genres input through the terminal. For example, if the user inputs "eggs and bacon" and selects "Japanese food" as a preference, the server generates a prompt based on this information.

[1020] Examples of specific prompts are as follows:

[1021] I have eggs and bacon in the fridge. Can you suggest some Japanese recipes?

[1022] The server sends this prompt to a generative AI model (e.g., GPT-4). The generative AI model analyzes the prompt and generates an appropriate recipe. For example, a recipe for "Japanese-style scrambled eggs with eggs and bacon" is generated.

[1023] Next, the server creates a shopping list based on the recipe, listing the condiments and vegetables needed to accompany the meal. For example, the following shopping list is created:

[1024] soy sauce

[1025] Mirin

[1026] spinach

[1027] The server then sends this shopping list via the API of the online supermarket the user uses, allowing the user to purchase the ingredients they need all at once via the online supermarket's application or website.

[1028] Furthermore, if the user inputs "I'm feeling stressed" into the emotion engine, the server can send an additional prompt to the generative AI model, suggesting a relaxing drink (e.g., "chamomile tea").

[1029] Examples of specific prompts are as follows:

[1030] There are eggs and bacon in the fridge. Can you suggest some Japanese recipes? Also, the user is feeling stressed so can you suggest some drinks that will help them relax?

[1031] In this way, it is possible to suggest recipes based on the contents of the user's refrigerator and preferences, and even meal suggestions based on the user's emotional state. It also makes it possible to create a shopping list and complete the purchase process at the online supermarket in one go, improving user convenience.

[1032] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1033] Step 1:

[1034] The user inputs the contents of the refrigerator and their favorite cuisine genre.

[1035] The user uses an application on a device (smartphone or PC) to input information about ingredients in the refrigerator (e.g. eggs and bacon) and their preferred cuisine (e.g. Japanese food). The input data is sent to the server.

[1036] Input: contents of fridge (eggs, bacon), favorite food genre (Japanese food)

[1037] Output: User input data sent to the server

[1038] Step 2:

[1039] The server receives the input data and sends prompts to the generative AI model.

[1040] The server receives the ingredient information and cuisine genre sent by the user, then generates a prompt based on this information and sends it to the generative AI model.

[1041] Specific prompt examples:

[1042] I have eggs and bacon in the fridge. Can you suggest some Japanese recipes?

[1043] Input: User input data (eggs, bacon, Japanese food)

[1044] Output: The prompt sent to the generative AI model

[1045] Step 3:

[1046] The generative AI model generates a recipe and returns it to the server.

[1047] The generative AI model receives the prompt and generates an appropriate recipe, for example, "Japanese-style scrambled eggs with eggs and bacon." The generated recipe is returned to the server.

[1048] Input: Prompt statement

[1049] Output: Generated recipe (Japanese-style scrambled eggs with eggs and bacon)

[1050] Step 4:

[1051] The server creates a shopping list based on the recipe.

[1052] The server analyzes the recipe returned by the generative AI model and lists the necessary seasonings, side dishes, etc. This creates a shopping list.

[1053] A concrete shopping list example:

[1054] soy sauce

[1055] Mirin

[1056] spinach

[1057] Input: Generated recipe

[1058] Output: Shopping list

[1059] Step 5:

[1060] The server sends the shopping list to the online supermarket.

[1061] The server sends the created shopping list via the API of the online supermarket the user uses, allowing the user to easily purchase the ingredients they need from the online supermarket.

[1062] Input: Shopping list

[1063] Output: Shopping list sent to online supermarket

[1064] Step 6:

[1065] The user purchases the necessary ingredients from an online supermarket.

[1066] Users can check the shopping list sent from the server via the online supermarket's application or website and purchase the ingredients they need all at once.

[1067] Input: Shopping list sent to online supermarket

[1068] Output: Purchased ingredients

[1069] (Application example 2)

[1070] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1071] Conventional recipe suggestion systems can suggest recipes based on the contents of the user's refrigerator and preferences, but they have a problem in that they cannot suggest recipes that take into account the user's emotional state. In addition, there is no mechanism for easily purchasing the ingredients needed for the suggested recipes, which results in low user convenience.

[1072] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1073] In this invention, the server includes a means for grasping the contents of the user's refrigerator and each user's preferences, a means for suggesting recipes based on the grasped information, a means for creating a shopping list according to the suggested recipes, a means for grasping the user's emotional state, a means for suggesting drinks with a relaxing effect according to the emotional state, and a means for transmitting the created shopping list to an online supermarket and providing a path for the user to make direct purchases. This enables recipe suggestions that take the user's emotional state into account and easy purchase of the necessary ingredients.

[1074] "User" means an individual or organization that uses the System.

[1075] "Contents of the refrigerator" refers to the food and beverages stored in the user's refrigerator.

[1076] "Preferences" refers to the preferences or tastes a user has for particular ingredients or dishes.

[1077] A "recipe" is a list of instructions and ingredients for making a particular dish.

[1078] A "shopping list" is a list of ingredients, seasonings, etc. that a user plans to purchase.

[1079] An "emotional state" is the emotion or mood a user is feeling at a particular point in time.

[1080] A "relaxing drink" is a drink that has the effect of reducing stress and relaxing the user.

[1081] An "online supermarket" is an online store that sells food and daily necessities via the Internet.

[1082] A "path" is the procedure or flow that a user follows to perform a specific operation or action through a system.

[1083] A system for implementing this invention includes means for grasping the contents of a user's refrigerator and each user's preferences, means for suggesting recipes based on the grasped information, means for creating a shopping list based on the suggested recipes, means for grasping the user's emotional state, means for suggesting drinks that have a relaxing effect according to the emotional state, and means for transmitting the created shopping list to an online supermarket and providing a path for the user to make direct purchases.

[1084] Hardware and Software Configuration

[1085] Hardware:

[1086] Smartphone

[1087] server

[1088] software:

[1089] Python

[1090] The Requests library (for sending HTTP requests)

[1091] Database management system (e.g. MySQL)

[1092] Emotion recognition engine (e.g. IBM Watson)

[1093] Data processing and calculation

[1094] Understanding what's in your fridge and what you like:

[1095] Through a smartphone application, users input the ingredients they have in their fridge and their preferences, and this data is sent to a server and stored in a database.

[1096] Recipe suggestions:

[1097] The server uses a generative AI model to suggest appropriate recipes based on the stored ingredients and preference data. For example, if a user inputs "eggs" and "bacon" and selects "Japanese food" as their preference, the system will suggest a recipe for "Japanese-style scrambled eggs with eggs and bacon."

[1098] Emotional state recognition and relaxing drink suggestions:

[1099] Users input their current emotional state through the emotion recognition engine. For example, if they input that they are "stressed," the system will suggest "chamomile tea," which has a relaxing effect.

[1100] Create and send a shopping list:

[1101] Based on the suggested recipe, a shopping list is automatically generated, including the necessary condiments and accompanying vegetables, and this list is sent through the online supermarket's API for easy purchase by the user.

[1102] Examples

[1103] If a user inputs that they have "eggs" and "bacon" in the fridge, selects "Japanese food" as their preference, and inputs "feeling stressed" as their emotional state, the system will suggest a recipe for "Japanese-style scrambled eggs with eggs and bacon" and add "chamomile tea" to the shopping list.

[1104] Example prompt for a generative AI model:

[1105] If a user inputs that they have eggs and bacon in the fridge, selects Japanese food as their preference, and indicates that they are stressed as their emotional state, suggest suitable recipes, add the necessary condiments and vegetable garnishes to their shopping list, and also suggest drinks that will have a relaxing effect.

[1106] In this way, personalized meal suggestions and shopping lists can be generated according to the user's needs.

[1107] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1108] Step 1:

[1109] The user launches a smartphone application and enters the ingredients they have in their fridge and their personal preferences.

[1110] Input: Ingredients (e.g. eggs, bacon), Preferences (e.g. Japanese food)

[1111] Output: The input data is sent to the server.

[1112] Specific behavior: The user enters ingredients and preferences into the application's input form, presses the "Submit" button, and the data is sent to the server.

[1113] Step 2:

[1114] The server stores the received data in a database.

[1115] Input: User-submitted ingredient and preference data

[1116] Output: Ingredients and preferences stored in a database

[1117] Specific operation: The server analyzes the received data and stores it in the corresponding table in the database.

[1118] Step 3:

[1119] Based on the data stored on the server, the server uses a generative AI model to suggest appropriate recipes.

[1120] Input: Ingredients and preferences stored in a database

[1121] Output: Suggested recipe (e.g. Japanese-style scrambled eggs with bacon)

[1122] Specific operation: The server inputs the prompt sentence into the generative AI model, generates an appropriate recipe, and returns the generated recipe to the user.

[1123] Step 4:

[1124] The user inputs their current emotional state through an emotion recognition engine.

[1125] Input: Emotional state (e.g. feeling stressed)

[1126] Output: The input data is sent to the server.

[1127] Specific behavior: When a user enters their emotional state into the application's emotion input form and presses the "Submit" button, the data is sent to the server.

[1128] Step 5:

[1129] The server suggests relaxing drinks based on the emotional state received.

[1130] Input: Emotional state data submitted by the user

[1131] Output: Suggested drink (e.g. chamomile tea)

[1132] What it does: The server analyzes your emotional state and suggests relaxing drinks.

[1133] Step 6:

[1134] The server generates a shopping list based on the suggested recipe and drink, including the necessary condiments and vegetable side dishes.

[1135] Input: Suggested recipe and drink

[1136] Output: Shopping list (e.g. soy sauce, mirin, green onion, chamomile tea)

[1137] What happens: The server parses the proposed recipe and drink and lists the required ingredients.

[1138] Step 7:

[1139] The server sends the generated shopping list to the online supermarket's API, allowing the user to make purchases directly.

[1140] Input: Shopping list

[1141] Output: Order status to online supermarket

[1142] Specific behavior: The server sends the shopping list to the online supermarket's API endpoint and checks whether the order was processed successfully.

[1143] In this way, personalized meal suggestions and shopping lists can be generated according to the user's needs.

[1144] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1145] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1146] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1147] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1148] [Third embodiment]

[1149] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1150] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1151] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1152] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1153] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1154] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1155] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1156] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1157] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1158] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1159] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1160] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1161] "Example 1"

[1162] In one embodiment of the present invention, a user inputs the contents of their refrigerator into a smartphone application. The user's preferences are also understood from their past recipe selection history and from direct input. Based on this information, the system suggests recipes to the user. Based on the suggested recipes, the system automatically creates a shopping list. The created shopping list is sent to the online supermarket that the user uses, and the user can make purchases directly based on the list.

[1163] "Example 2"

[1164] As a specific example, if a user inputs that they have eggs and bacon in their fridge and selects "Japanese food" as their preference, the system will suggest a recipe for "Japanese-style scrambled eggs with eggs and bacon." The system will then add the seasonings and accompanying vegetables needed for that recipe to a shopping list, which will be sent to the online supermarket the user uses. The user can then use that list to purchase the ingredients they need all at once.

[1165] The process flow of each embodiment will be described below.

[1166] "Example 1"

[1167] Step 1: The user opens the application on their smartphone and enters the contents of their refrigerator.

[1168] Step 2: The user inputs their preferences (e.g. Japanese, Western, Chinese, etc.) into the application.

[1169] Step 3: The system suggests recipes based on what's in the user's fridge and their preferences.

[1170] Step 4: The system automatically creates a shopping list based on the suggested recipes.

[1171] Step 5: The system sends the shopping list created by the user to the online supermarket that the user uses.

[1172] Step 6: The user purchases ingredients from the online supermarket based on the shopping list.

[1173] "Example 2"

[1174] Step 1: The user tells the application that they have eggs and bacon in the fridge.

[1175] Step 2: The user enters "Japanese food" as a preference into the application.

[1176] Step 3: The system suggests a recipe for "Japanese-style scrambled eggs with eggs and bacon."

[1177] Step 4: The system adds the condiments and vegetables needed for the recipe to a shopping list.

[1178] Step 5: The system sends the shopping list created by the user to the online supermarket that the user uses.

[1179] Step 6: The user purchases ingredients from the online supermarket based on the shopping list.

[1180] Example 1

[1181] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1182] In the conventional system, the process of users managing the contents of their refrigerator, finding recipes that matched their tastes, and purchasing the necessary ingredients was complicated. In addition, users would create shopping lists without considering the ingredients they already had, which could lead to unnecessary purchases. Furthermore, the purchasing process at online supermarkets was done manually, which was time-consuming and cumbersome.

[1183] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1184] In this invention, the server includes a means for a user to input the contents of the refrigerator, a means for grasping the user's preferences, a means for proposing recipes using a generative AI model based on the grasped information, a means for automatically creating a shopping list based on the proposed recipes, a means for transmitting the created shopping list to an online supermarket, and a means for providing a path for the user to directly purchase at the online supermarket. This enables the user to efficiently manage the contents of the refrigerator, easily find recipes that suit their preferences, and purchase the necessary ingredients without waste.

[1185] "Means for users to input the contents of their refrigerator" refers to an interface that allows users to input the ingredients in their refrigerator using a smartphone or other device.

[1186] "Means of understanding user preferences" refers to a function for collecting and storing a user's food preferences based on the user's past recipe selection history or direct input.

[1187] "Means of suggesting recipes using a generative AI model" refers to a function that uses a generative AI model to generate and suggest appropriate recipes based on data on the contents of the user's refrigerator and their preferences.

[1188] The "means for automatically generating a shopping list" is a function for automatically generating a list of ingredients needed based on a suggested recipe.

[1189] "Means for sending a shopping list to an online supermarket" is a function for sending the created shopping list to the online supermarket used by the user.

[1190] "Means of providing a path for users to make purchases directly from online supermarkets" refers to a function that enables users to purchase groceries directly on the online supermarket platform based on the shopping list submitted to them.

[1191] The present invention is a system that allows a user to manage the contents of a refrigerator, suggest recipes that suit the user's preferences, and purchase the necessary ingredients efficiently. A specific embodiment of this system will be described below.

[1192] System Configuration

[1193] Hardware and Software

[1194] User device: Mobile device such as smartphone or tablet, using applications that run on iOS and Android.

[1195] Server: Located on the cloud, it hosts the database and generative AI model. It uses MySQL for the database and OpenAI's GPT-4 for the generative AI model.

[1196] Entering and Saving Data

[1197] The user enters the contents of the refrigerator

[1198] A user opens the smartphone application and inputs the ingredients in the refrigerator, either manually, by voice, by scanning a barcode, etc. For example, a user inputs "chicken, carrots, onions."

[1199] Understand user preferences

[1200] The user's preferences are captured from their past recipe selection history or through direct input. For example, if a user has previously selected "curry" as their favorite dish, that information is also stored in the database.

[1201] Recipe Suggestions

[1202] Server suggests recipes

[1203] The server uses a generative AI model to suggest recipes based on the contents of the user's fridge and their preferences. Specifically, it sends the following prompt to the generative AI model:

[1204] Contents of user's fridge: Chicken, carrots, onions

[1205] User preference: Curry

[1206] Generate suggested recipes.

[1207] The generative AI model generates a recipe for "chicken curry" based on this prompt, and the server sends the recipe to the user's smartphone.

[1208] Create and send a shopping list

[1209] The server creates the shopping list.

[1210] When a user selects a suggested recipe, the server automatically creates a shopping list of ingredients needed for that recipe, for example, "chicken, carrots, onions, curry roux, and any other necessary seasonings."

[1211] The server sends the shopping list to the online supermarket.

[1212] The server then sends the created shopping list to the online supermarket the user uses, using an API to link with the online supermarket's system.

[1213] Making a purchase

[1214] The user purchases from the online supermarket

[1215] The user opens the online supermarket's application or website, checks the shopping list sent from the server, and can then directly purchase the ingredients they need based on that list.

[1216] Examples

[1217] For example, if a user inputs that they have "chicken, carrots, and onions" in their fridge and has previously preferred "curry," the system will suggest a recipe for "chicken curry." Here is an example of a prompt to be input to the generative AI model:

[1218] Contents of user's fridge: Chicken, carrots, onions

[1219] User preference: Curry

[1220] Generate suggested recipes.

[1221] Based on this prompt, the generative AI model generates a recipe for "chicken curry" and suggests it to the user. Based on the suggested recipe, the system creates a shopping list of "chicken, carrots, onions, curry roux, and other necessary seasonings" and sends it to the online supermarket. The user can use this list to directly purchase the necessary ingredients.

[1222] In this way, users can efficiently manage the contents of their refrigerator, easily find recipes that suit their tastes, and buy the ingredients they need without waste.

[1223] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1224] Step 1:

[1225] The user enters the contents of the refrigerator

[1226] The user opens the smartphone application and inputs the ingredients in the refrigerator. Input methods include manual input, voice input, barcode scanning, etc. For example, the user inputs "chicken, carrots, onions." This input data is stored in a database on the cloud. The input data includes the names of ingredients and their quantities.

[1227] Step 2:

[1228] Understand user preferences

[1229] A user's preferences are understood from past recipe selection history or direct input. For example, if a user has previously selected "curry" as their favorite food, that information is also stored in the database. The server analyzes the user's selection history and extracts preference trends. The input data is the past recipe selection history or direct input of the user's preferences, and the output data is the user's preference trends.

[1230] Step 3:

[1231] Server suggests recipes

[1232] The server uses a generative AI model to suggest recipes based on the contents of the user's fridge and their preferences. Specifically, it sends the following prompt to the generative AI model:

[1233] Contents of user's fridge: Chicken, carrots, onions

[1234] User preference: Curry

[1235] Generate suggested recipes.

[1236] The generative AI model generates a recipe for "chicken curry" based on this prompt, and the server sends the recipe to the user's smartphone. The input data is the contents of the refrigerator and the user's preferences, and the output data is the suggested recipe.

[1237] Step 4:

[1238] The server creates the shopping list.

[1239] When a user selects a suggested recipe, the server automatically creates a shopping list of ingredients needed based on the recipe. For example, it generates a list of "chicken, carrots, onions, curry roux, and other necessary seasonings." The server also provides a function to remove ingredients from the list that the user already has. The input data is the selected recipe and the contents of the refrigerator, and the output data is the shopping list.

[1240] Step 5:

[1241] The server sends the shopping list to the online supermarket.

[1242] The server sends the created shopping list to the online supermarket that the user uses. An API is used to send the list, linking with the online supermarket's system. The input data is the shopping list and the online supermarket's API information, and the output data is the shopping list sent to the online supermarket.

[1243] Step 6:

[1244] The user purchases from the online supermarket

[1245] The user opens the online supermarket application or website and checks the shopping list sent from the server. The user can directly purchase the ingredients he needs based on the list. The input data is the shopping list sent to the online supermarket, and the output data is the purchased ingredients.

[1246] (Application example 1)

[1247] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1248] Conventional refrigerator management systems had the problem that users had to manually manage the contents of their refrigerators, search for recipes themselves, and list the ingredients they needed to purchase, which was a lot of work. In addition, it was difficult to suggest recipes that matched the user's preferences, and creating shopping lists and purchasing procedures were cumbersome. This reduced user convenience and made efficient ingredient management difficult.

[1249] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1250] In this invention, the server includes a means for grasping the contents of the user's refrigerator and each user's preferences, a means for proposing recipes based on the grasped information, a means for creating a shopping list according to the proposed recipes, and a means for transmitting the created shopping list to an electronic commerce platform and providing a path for the user to make direct purchases. This enables the user to efficiently manage the contents of the refrigerator, receive recipe suggestions according to their preferences, and easily purchase the necessary ingredients.

[1251] A "user" is an individual who uses the system to input the contents of their refrigerator and their preferences and receive recipe suggestions and shopping lists.

[1252] "What's in the refrigerator" refers to information about the food and beverages currently stored in the user's refrigerator.

[1253] "Preferences" are the ingredients and cooking preferences that a user indicates based on their past recipe selection history or direct input.

[1254] A "recipe" is a list of instructions and ingredients needed to make a particular dish.

[1255] A "shopping list" is a list of ingredients and products to purchase that is created based on a suggested recipe.

[1256] An "e-commerce platform" is an online marketplace for purchasing goods and services via the Internet.

[1257] A "guideline" is the sequence of steps or operations that allow a user to directly purchase items based on their shopping list.

[1258] A "generative AI model" is an artificial intelligence algorithm used to suggest optimal recipes based on what's in a user's fridge and their preferences.

[1259] A "prompt sentence" is an instruction sentence for suggesting recipes or creating a shopping list that is generated by a generative AI model based on information input by the user.

[1260] A system for implementing the present invention includes means for identifying the contents of a user's refrigerator and each user's preferences, means for suggesting recipes based on the identified information, means for creating a shopping list based on the suggested recipes, and means for transmitting the created shopping list to an e-commerce platform and providing a path for the user to make direct purchases.

[1261] System Programming

[1262] The server provides an interface for users to input the contents of their refrigerator using their smartphones. The user's preferences are understood from their past recipe selection history and direct input. This information is sent to the server and stored in a database.

[1263] The server uses a generative AI model to suggest optimal recipes based on the contents of the user's refrigerator and their preferences. The generative AI model generates prompts based on the user's input information and suggests recipes.

[1264] Based on the suggested recipes, the server automatically creates a shopping list, which is then sent to the e-commerce platform for users to purchase directly.

[1265] Hardware and Software Used

[1266] Hardware: Smartphones, servers

[1267] Software: Python, Requests library, generative AI models

[1268] Data processing and calculation

[1269] 1. User inputs the contents of the refrigerator: The user inputs the contents of the refrigerator using a smartphone application. This information is sent to the server and stored in a database.

[1270] 2. Understanding user preferences: User preferences are understood from past recipe selection history and direct input. This information is also sent to the server and stored in the database.

[1271] 3. Recipe Suggestion: The server uses a generative AI model to suggest optimal recipes based on the contents of the user's refrigerator and their preferences. The generative AI model generates prompts based on the user's input information and makes recipe suggestions.

[1272] 4. Create shopping list: Based on the suggested recipes, the server automatically creates a shopping list.

[1273] 5. Send shopping list: Once created, the shopping list is sent to the e-commerce platform to allow the user to make purchases directly.

[1274] Examples

[1275] Suppose a user inputs that they have “Ingredient A” and “Ingredient B” in their fridge, and that they prefer “vegetarian” and “non-spicy” food. If they have previously selected “Recipe A” and “Recipe B,” the system will suggest “Recipe C” and “Recipe D.” It will then generate a shopping list that includes “Ingredient 1” and “Ingredient 2” and send it to the e-commerce platform.

[1276] Examples of prompt statements

[1277] User ID: 12345

[1278] Contents of the fridge: ["ingredient A", "ingredient B"]

[1279] User Preferences: {"Vegetarian": True, "Spicy": False}

[1280] Past recipe selection history: ["Recipe A", "Recipe B"]

[1281] Suggested recipes: ["Recipe C", "Recipe D"]

[1282] Generated shopping list: ["ingredient 1", "ingredient 2"]

[1283] Sending status to online supermarket: 200

[1284] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1285] Step 1:

[1286] The user inputs the contents of the refrigerator using a smartphone application.

[1287] Input: Information about ingredients in the user's refrigerator (e.g., ingredient A, ingredient B)

[1288] Data processing: The entered ingredient information is converted into JSON format and sent to the server.

[1289] Output: The contents of the fridge sent to the server.

[1290] Step 2:

[1291] The server knows the user's preferences.

[1292] Input: User's past recipe selection history and direct preference input (e.g. vegetarian, not spicy)

[1293] Data processing: Store user preference information in a database.

[1294] Output: User preferences stored in a database

[1295] Step 3:

[1296] The server suggests recipes using a generative AI model.

[1297] Input: Fridge contents data, user preferences data

[1298] Data processing: A generative AI model uses this data to generate prompts and suggest optimal recipes.

[1299] Output: Suggested recipes (e.g. Recipe C, Recipe D)

[1300] Step 4:

[1301] The server creates a shopping list based on the suggested recipes.

[1302] Input: Suggested recipe data

[1303] Data processing: List ingredients needed for recipes and generate shopping lists.

[1304] Output: The created shopping list (e.g. ingredient 1, ingredient 2)

[1305] Step 5:

[1306] The server transmits the created shopping list to the e-commerce platform.

[1307] Input: The shopping list data created

[1308] Data processing: Formatting the shopping list to send to the e-commerce platform's API.

[1309] Output: Status of submission to e-commerce platform (e.g. 200 OK)

[1310] Step 6:

[1311] A user purchases products based on a shopping list through an e-commerce platform.

[1312] Input: Shopping list submitted to e-commerce platform

[1313] Data processing: The e-commerce platform searches for products based on the shopping list and completes the purchase.

[1314] Output: A confirmation message for the purchase (e.g. Purchase Completed)

[1315] Example 2

[1316] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1317] Conventional recipe suggestion systems would only suggest appropriate recipes when users input the ingredients they had in their fridge and their preferred cooking genre, but lacked the functionality to automatically list the necessary seasonings and accompaniments and send them to the online supermarket. As a result, users had to manually create a shopping list based on the suggested recipes and then go through the purchasing process separately at the online supermarket. This process was time-consuming and inefficient.

[1318] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1319] In this invention, the server includes a means for grasping the contents of a user's refrigerator and each user's preferences, a means for proposing a recipe using a generative AI model based on the grasped information, a means for creating a shopping list including necessary seasonings and accompaniment ingredients based on the proposed recipe, a means for transmitting the created shopping list to an online supermarket, and a means for providing a path for the user to purchase the necessary ingredients in one go through the online supermarket. As a result, the user is automatically suggested appropriate recipes based on the contents of the refrigerator and their preferences, and is able to efficiently purchase the necessary ingredients.

[1320] "User" refers to an individual who uses the system to input the contents of their refrigerator and their preferred cuisine genres.

[1321] "What's in the refrigerator" refers to the food items currently stored in the user's refrigerator.

[1322] "Favorite Cuisine Genre" refers to a particular style or type of cuisine selected by a User.

[1323] A "generative AI model" refers to an artificial intelligence algorithm that generates an appropriate recipe based on input information.

[1324] A "prompt sentence" refers to an input sentence used to request a generative AI model to generate a recipe.

[1325] A "recipe" is a list of instructions or ingredients for making a particular dish.

[1326] "Condiments" refer to ingredients used to adjust the flavor of food.

[1327] "Accompaniment ingredients" refers to additional ingredients served with the main dish.

[1328] A "shopping list" refers to a list of ingredients needed based on a suggested recipe.

[1329] An "online supermarket" refers to a website or application that provides the service of purchasing food and daily necessities online.

[1330] "Path" refers to the steps and flow that users take to purchase the ingredients they need in one go through an online supermarket.

[1331] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1332] This invention is a system that, when a user inputs the ingredients in the refrigerator and the type of cooking they like, suggests appropriate recipes, automatically creates a shopping list including the necessary seasonings and garnishes, and transmits it to an online supermarket. A specific embodiment of this system is described below.

[1333] 1. System Overview

[1334] The system involves a series of processes: identifying the contents of a user's refrigerator and preferred cooking genres, suggesting recipes using a generative AI model, creating a shopping list based on the suggested recipes, and sending it to the online supermarket.

[1335] 2. Hardware and Software Used

[1336] Hardware: The device used by the user (smartphone, tablet, computer, etc.)

[1337] Software: Server, generative AI model (e.g. OpenAI's GPT-4), online supermarket API

[1338] 3. Program Processing

[1339] 1. Receiving User Input

[1340] The user inputs into the device the ingredients they have in their fridge (e.g. eggs and bacon) and their preferred cuisine type (e.g. Japanese food).

[1341] The terminal transmits the input information to the server.

[1342] 2. Recipe suggestions

[1343] The server analyzes the received information on ingredients and cuisine genre.

[1344] The server sends prompts to the generative AI model, which then generates an appropriate recipe.

[1345] The server sends the generated recipe (e.g. "Japanese-style scrambled eggs with eggs and bacon") back to the terminal.

[1346] 3. Generate a shopping list

[1347] The server analyzes the generated recipe and lists the necessary condiments and side dishes.

[1348] The server sends this shopping list to the API of the online supermarket the user is using.

[1349] 4. Purchase procedure

[1350] The user accesses the online supermarket's website through their device and checks the submitted shopping list.

[1351] Users purchase the ingredients they need in bulk.

[1352] 4. Specific Examples

[1353] The user types into the device, "I have eggs and bacon in the fridge," and selects "Japanese food" as their preferred cuisine genre.

[1354] The server uses a generative AI model to suggest a recipe for "Japanese-style scrambled eggs with eggs and bacon."

[1355] The server adds the necessary seasonings (e.g. soy sauce, mirin) and accompanying vegetables (e.g. spinach) to a shopping list and sends it to the online supermarket.

[1356] Users check their shopping list on the online supermarket website and purchase all the ingredients they need in one go.

[1357] 5. Examples of prompts

[1358] I have eggs and bacon in the fridge. Can you suggest some Japanese recipes?

[1359] The above is a specific embodiment for carrying out the present invention. This system automatically suggests suitable recipes to users based on the contents of their refrigerator and their preferences, enabling them to efficiently purchase the ingredients they need.

[1360] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1361] Program processing flow

[1362] Step 1: Receiving User Input

[1363] The user inputs into the device the ingredients they have in their fridge (e.g. eggs and bacon) and their preferred cuisine type (e.g. Japanese food).

[1364] Input: A user types "eggs and bacon" into a terminal application and selects "Japanese food" from a drop-down menu.

[1365] The terminal transmits the input information to the server.

[1366] Output: The data sent from the terminal to the server is ingredient information and cuisine genre information.

[1367] Specific actions: A user opens an application on their smartphone, types "eggs and bacon" into the text box, selects "Japanese food" from the drop-down menu, and presses the "Submit" button.

[1368] Step 2: Suggest a recipe

[1369] The server analyzes the received information on ingredients and cuisine genre.

[1370] Input: Ingredient information and cuisine genre information received by the server from the terminal.

[1371] The server sends prompts to the generative AI model, which then generates an appropriate recipe.

[1372] Data processing: The server sends a prompt to the generative AI model: "There are eggs and bacon in the fridge. Please suggest a Japanese recipe."

[1373] Output: Recipe information returned from the generative AI model.

[1374] The server sends the generated recipe (e.g. "Japanese-style scrambled eggs with eggs and bacon") back to the terminal.

[1375] Specific operation: The server sends a prompt to the generative AI model and displays the returned recipe on the user's device.

[1376] Step 3: Generate a shopping list

[1377] The server analyzes the generated recipe and lists the necessary condiments and side dishes.

[1378] Input: The generated recipe information.

[1379] Data processing: The server extracts the necessary seasonings (e.g. soy sauce, mirin) and accompanying vegetables (e.g. spinach) from the recipe.

[1380] Output: A list of ingredients needed.

[1381] The server sends this shopping list to the API of the online supermarket the user is using.

[1382] Specific behavior: The server sends a request to the online supermarket's API saying, "Please add these items to your shopping list."

[1383] Step 4: Checkout

[1384] The user accesses the online supermarket's website through their device and checks the submitted shopping list.

[1385] Input: The shopping list sent by the server.

[1386] Output: Shopping list displayed on the online supermarket website.

[1387] Users purchase the ingredients they need in bulk.

[1388] Specific behavior: The user opens the online supermarket app, goes to the "Shopping List" section, checks the list sent from the server, and presses the "Purchase" button.

[1389] The above is the specific processing flow of the program of this system.

[1390] (Application example 2)

[1391] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1392] Conventional recipe suggestion systems were insufficient in suggesting recipes that took into account the ingredients in the user's refrigerator and preferences, and had problems with creating a shopping list based on the suggested recipe and the complicated process of purchasing the necessary ingredients. In addition, there was no way for users to easily purchase additional ingredients required for the suggested recipe, which resulted in low convenience. There is a need to solve these problems and improve user convenience.

[1393] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1394] In this invention, the server includes a means for grasping the contents of a user's refrigerator and each user's preferences, a means for proposing recipes using a generative AI model based on the grasped information, a means for creating a shopping list according to the proposed recipes, and a means for transmitting the created shopping list to a food delivery service and providing a path for the user to directly purchase. This allows the user to receive recipe suggestions based on the ingredients in the refrigerator and their preferences, and further allows the user to easily purchase any additional ingredients that are required.

[1395] A "user" is an individual who uses the system to input the contents of their refrigerator and their preferences and receive recipe suggestions and shopping lists.

[1396] "Contents of the refrigerator" refers to ingredients and foods currently stored in the user's refrigerator.

[1397] "Preferences" refers to a user's personal preferences for a particular cuisine genre or seasoning.

[1398] A "generative AI model" is an artificial intelligence model that generates appropriate recipes based on user input information.

[1399] A "recipe" is a set of instructions that lists the steps and ingredients needed to make a particular dish.

[1400] A "shopping list" is a list of ingredients and seasonings needed based on a suggested recipe.

[1401] A "food delivery service" is a service that delivers ingredients and meals ordered online by users to their homes.

[1402] A "guideline" refers to the flow of procedures and operations that allow users to easily purchase the ingredients they need through the system.

[1403] A system for implementing this invention includes means for understanding the contents of a user's refrigerator and their individual preferences, means for suggesting recipes using a generative AI model based on the understood information, means for creating a shopping list based on the suggested recipes, and means for sending the created shopping list to a food delivery service and providing a path for the user to make direct purchases.

[1404] Hardware and Software Configuration

[1405] Hardware: Smartphones, servers

[1406] Software: Python, OpenAI API, HTTP request library (requests)

[1407] Data processing and calculation

[1408] 1. Receiving user input: The user uses their smartphone to input the ingredients they have in their fridge and their preferred cuisine, for example, "eggs, bacon" and "Japanese food."

[1409] 2. Using a generative AI model: The server uses the OpenAI API to generate a recipe based on the user's input. An example of a prompt is as follows:

[1410] Ingredients in the fridge: eggs, bacon

[1411] Favorite food genre: Japanese food

[1412] Suggested recipe:

[1413] 3. Creating a shopping list: The server extracts the additional ingredients required from the generated recipe and creates a shopping list. For example, "soy sauce, mirin, and green onion" are added.

[1414] 4. Sending an order to a food delivery service: The server sends the created shopping list to the food delivery service and provides a path for the user to purchase directly. This allows the user to easily purchase the ingredients they need.

[1415] Examples

[1416] If a user inputs that they have "eggs and bacon" in their fridge and selects "Japanese" as their preferred cuisine genre, the server will use the generative AI model to suggest a recipe for "Japanese-style scrambled eggs with eggs and bacon." It will then add "soy sauce, mirin, and green onions" as additional ingredients needed to a shopping list and send this list to a food delivery service. The user can easily purchase these ingredients and have them delivered via their smartphone.

[1417] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1418] Step 1:

[1419] The user uses a smartphone to input the ingredients in the refrigerator and the type of cuisine they prefer.

[1420] Input: Ingredients in the fridge (e.g., "eggs, bacon"), favorite cuisine (e.g., "Japanese food")

[1421] Output: User input data

[1422] Specific actions: A user opens the application on their smartphone, enters ingredients in the fridge into a text box, and selects their preferred cuisine from a drop-down menu.

[1423] Step 2:

[1424] The server receives the user input data and generates the recipe using a generative AI model.

[1425] Input: User-entered data (ingredients in the fridge, favorite cuisine genres)

[1426] Output: Suggested recipe

[1427] Specific operation: The server calls the OpenAI API and sends the following prompt sentence to the generative AI model.

[1428] Ingredients in the fridge: eggs, bacon

[1429] Favorite food genre: Japanese food

[1430] Suggested recipe:

[1431] The generative AI model generates a recipe based on this prompt and returns it to the server.

[1432] Step 3:

[1433] The server extracts any additional ingredients required from the generated recipe and creates a shopping list.

[1434] Input: Suggested recipe

[1435] Output: Shopping list

[1436] Specific behavior: The server parses the generated recipe text and extracts any additional ingredients needed (e.g., "soy sauce, mirin, green onion"), organizes these ingredients into a list, and creates a shopping list.

[1437] Step 4:

[1438] The server creates a shopping list and sends it to the food delivery service, providing a path for the user to make a purchase directly.

[1439] Input: Shopping list

[1440] Output: Order data for food delivery service

[1441] What happens: The server calls the food delivery service's API and sends the shopping list as order data, including the user's address information, to ensure that the order is processed successfully.

[1442] Step 5:

[1443] The user receives an order confirmation from the food delivery service via smartphone and purchases the necessary ingredients.

[1444] Input: Order confirmation from food delivery service

[1445] Output: Purchased ingredients

[1446] How it works: The user receives an order confirmation on their smartphone app, waits for the delivery of the ingredients they need, and then collects the purchased ingredients when the delivery arrives.

[1447] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1448] "Example 1"

[1449] The present invention is a system including a means for grasping the contents of a user's refrigerator and each user's preferences, a means for suggesting recipes based on the grasped information, a means for creating a shopping list based on the suggested recipes, and an emotion engine for recognizing the user's emotions. The emotion engine has a function for adjusting the recipe suggestions and the shopping list according to the user's emotions. For example, when the user is feeling stressed, the emotion engine suggests recipes that include ingredients that have a relaxing effect. Also, when the user is feeling happy, the emotion engine suggests special recipes suitable for celebrations. This makes it possible to suggest meals according to the user's emotional state.

[1450] "Example 2"

[1451] As a concrete example, consider the case where a user inputs to the application that they have eggs and bacon in the fridge and selects "Japanese food" as their preference. Furthermore, suppose the user inputs to the emotion engine that they "feel stressed." In this case, the system will suggest a recipe for "Japanese-style scrambled eggs with eggs and bacon" and create a shopping list that adds "chamomile tea," which has a relaxing effect. This makes it possible to suggest meals according to the user's emotional state.

[1452] The process flow of each embodiment will be described below.

[1453] "Example 1"

[1454] Step 1: The user enters the contents of the fridge and each person's preferences into the application.

[1455] Step 2: The system will understand the information you entered and suggest recipes based on it.

[1456] Step 3: The system creates a shopping list with suggested recipes.

[1457] Step 4: The user inputs their current emotional state into the emotion engine.

[1458] Step 5: The emotion engine adapts recipe suggestions and shopping lists based on the user's emotions.

[1459] "Example 2"

[1460] Step 1: The user tells the application what eggs and bacon they have in the fridge, and that their preference is "Japanese food."

[1461] Step 2: The user types "I'm stressed" into the emotion engine.

[1462] Step 3: The system understands the input information and emotional state and suggests a recipe for "Japanese-style scrambled eggs with eggs and bacon."

[1463] Step 4: The system creates a shopping list to which it adds the relaxing "chamomile tea."

[1464] Example 1

[1465] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1466] Conventional recipe suggestion systems have problems in that it is difficult to suggest recipes that fully take into account the contents of the user's refrigerator and preferences, and they are unable to suggest recipes or adjust shopping lists according to the user's emotional state. Furthermore, creating a shopping list based on the suggested recipes and sending it to the online supermarket is done manually, which is time-consuming for users.

[1467] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1468] In this invention, the server includes means for inputting the contents of the user's refrigerator, means for grasping the user's preferences, means for suggesting recipes based on the grasped information, means for automatically creating a shopping list according to the suggested recipes, means for transmitting the created shopping list to the online supermarket, and means for recognizing the user's emotional state and adjusting the recipe suggestions and shopping list accordingly. This makes it possible to automatically create optimal recipe suggestions and shopping lists according to the contents of the user's refrigerator, preferences, and emotional state, and to transmit them to the online supermarket.

[1469] "Means for inputting the contents of the user's refrigerator" refers to an interface that allows a user to input the ingredients in their refrigerator via a smartphone or other device.

[1470] The "means of understanding user preferences" refers to a system that stores in a database the recipes the user has previously chosen, as well as the ingredients and types of dishes that the user has registered as their preferences, in order to understand the user's tastes.

[1471] The "means of suggesting recipes based on the captured information" refers to an algorithm that searches for and suggests appropriate recipes based on the contents of the user's refrigerator and their preferences.

[1472] The "means for automatically creating a shopping list according to a proposed recipe" is a system that lists ingredients needed for a proposed recipe and automatically generates a shopping list.

[1473] "Means for sending the created shopping list to the online supermarket" refers to a system that sends the created shopping list via the API of the online supermarket used by the user, enabling the user to make purchases directly.

[1474] "Means for recognizing the user's emotional state and adjusting recipe suggestions and shopping lists accordingly" refers to a system that recognizes the user's emotional state through sensors and input data, and suggests and adjusts recipes and shopping lists according to that state.

[1475] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1476] This invention is a system that proposes optimal recipes based on the contents of a user's refrigerator, preferences, and emotional state, and automatically creates and transmits a shopping list. A specific embodiment of this system will be described below.

[1477] Input the contents of the user's refrigerator

[1478] Users use a smartphone application to input the contents of their refrigerator. The application runs on iOS and Android platforms. Users enter ingredients such as "chicken, cabbage, and carrots" by text into the application's input form, or scan the barcodes of the ingredients using the barcode scanner function. The input data is sent from the device to the server and stored in a database.

[1479] Understanding user preferences

[1480] Users input recipes they have previously chosen within the application, as well as ingredients and types of cuisine they have registered as preferences. For example, if a user inputs that they "like Japanese food," that information is sent from the device to the server and stored in the user preference database. The server uses this data to determine the user's preferences.

[1481] Recipe Suggestions

[1482] The server suggests appropriate recipes based on the contents of the refrigerator and the user's preferences, using an algorithm that references the recipe database and the user's preference database. For example, if a user has "chicken, cabbage, and carrots," the server will suggest "stir-fried chicken and cabbage" or "teriyaki chicken."

[1483] Automatic shopping list creation

[1484] The server automatically creates a shopping list based on the proposed recipe. Specifically, the ingredients required for the recipe are added to the list. For example, if a recipe for "stir-fried chicken with vegetables" is proposed, the necessary ingredients (chicken, cabbage, carrots, seasonings, etc.) are added to the list. This list is sent to the user's device and displayed within the application.

[1485] Send to online supermarket

[1486] When a user checks the shopping list and wishes to purchase from an online supermarket, a request is sent from the terminal to the server. The server sends the shopping list through the API of the online supermarket the user is using. For example, the list is sent to a service such as an online shopping platform. The user can then make a purchase directly based on the list.

[1487] Emotion engine adjustment

[1488] The server recognizes the user's emotional state using the emotion engine. Specifically, the user inputs their emotional state within the application, or detects emotions through the smartphone's sensors (camera and microphone). For example, if the user inputs that they are "feeling stressed," the emotion engine will suggest recipes that contain ingredients that have a relaxing effect. Also, if the user inputs that they are "feeling happy," the emotion engine will suggest special recipes suitable for celebrations.

[1489] Examples of concrete examples and prompts

[1490] If a user inputs that they have "chicken, cabbage, and carrots" in their fridge and has previously registered that they like "Japanese food," the server will suggest "chicken and cabbage stir fry" or "teriyaki chicken." If the emotion engine recognizes that the user is stressed, it will also suggest "relaxing herbal tea."

[1491] Example prompt:

[1492] "I have some chicken, cabbage, and carrots in the fridge. I like Japanese food. I'm feeling stressed. Can you recommend some recipes?"

[1493] In this way, based on the contents of the user's fridge, preferences, and emotional state, the server suggests optimal recipes and shopping lists.

[1494] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1495] Step 1:

[1496] The user launches the smartphone application and inputs the contents of the refrigerator. Input methods include text input and barcode scanner functions. When the user inputs "chicken, cabbage, carrots," the data is sent from the device to the server. The server stores the received data in a database. The input data is a list of ingredients, and the output is the ingredient information stored in the database.

[1497] Step 2:

[1498] Within the application, the user inputs previously selected recipes, favorite ingredients, and types of cuisine. For example, if the user inputs "I like Japanese food," that information is sent from the device to the server and stored in a user preference database. The server uses this data to understand the user's preferences. The input data is the user's preference information, and the output is the preference information stored in the database.

[1499] Step 3:

[1500] The server suggests appropriate recipes based on the contents of the refrigerator and preference data entered by the user. Specifically, the server searches the recipe database and extracts recipes that match the contents of the user's refrigerator. For example, if the user has "chicken, cabbage, and carrots," the server will suggest "stir-fried chicken and cabbage" or "teriyaki chicken." The input data are the contents of the refrigerator and preference information, and the output is the suggested recipe.

[1501] Step 4:

[1502] The server automatically creates a shopping list based on the proposed recipe. Specifically, the ingredients required for the recipe are added to the list. For example, if a recipe for "stir-fried chicken with vegetables" is proposed, the necessary ingredients (chicken, cabbage, carrots, seasonings, etc.) are added to the list. This list is sent to the user's device and displayed within the application. The input data is the proposed recipe, and the output is the shopping list.

[1503] Step 5:

[1504] If the user checks the shopping list and wishes to make a purchase at the online supermarket, a request is sent from the terminal to the server. The server sends the shopping list through the API of the online supermarket the user is using. For example, the list is sent to a service such as an online shopping platform. The user can then make a purchase directly based on the list. The input data is the shopping list, and the output is the result of sending it to the online supermarket.

[1505] Step 6:

[1506] The server recognizes the user's emotional state using the emotion engine. Specifically, the user inputs their emotional state within the application or detects emotions through the smartphone's sensors (camera and microphone). For example, if the user inputs that they are "feeling stressed," the emotion engine will suggest recipes that contain ingredients with a relaxing effect. Or, if the user inputs that they are "feeling happy," the emotion engine will suggest special recipes suitable for celebrations. The input data is the user's emotional state, and the output is the adjusted recipe suggestions.

[1507] (Application example 1)

[1508] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1509] Conventional recipe suggestion systems could suggest recipes based on the contents of the user's refrigerator and preferences, but had the problem of not being able to make suggestions that took the user's emotional state into account. Even if there was a function to create a shopping list based on the suggested recipes and send it to an online supermarket, it was difficult to increase user satisfaction because it could not flexibly respond to the user's emotions. Furthermore, there was a lack of functionality to provide a path for users to make direct purchases, which made it less convenient.

[1510] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1511] In this invention, the server includes a means for grasping the contents of the user's refrigerator and each user's preferences, a means for proposing recipes based on the grasped information, a means for creating a shopping list according to the proposed recipes, a means for recognizing the user's emotions and adjusting the recipe suggestions according to the recognized emotions, and a means for transmitting the created shopping list to an online supermarket and providing a path for the user to make direct purchases. This makes it possible to propose recipes and create shopping lists according to the user's emotional state, thereby increasing user satisfaction and convenience.

[1512] "Means for understanding the contents of the user's refrigerator" refers to a means for recording and managing the types and quantities of food and beverages in the user's refrigerator.

[1513] "Means for understanding individual preferences" refers to means for understanding and recording a user's food preferences and tastes based on the user's past selection history and information directly entered by the user.

[1514] The "means for suggesting recipes" refers to a means for presenting appropriate recipes to the user based on the contents of the refrigerator and the user's preferences.

[1515] A "means for creating a shopping list" is a means for listing ingredients and seasonings needed for a suggested recipe and for the user to clearly indicate the items they should purchase.

[1516] A "means for recognizing a user's emotions" is a means for analyzing emotions from a user's input and actions and understanding their emotional state.

[1517] A "means for adjusting recipe suggestions according to emotions" is a means for changing the content and suggestion method of a recipe based on the recognized emotional state of the user, and presenting recipes that are appropriate to the user's emotions.

[1518] The "means for sending to the online supermarket" refers to a means for sending the created shopping list to an online supermarket so that the user can directly carry out the purchase procedure based on the list.

[1519] "Means for providing a path to direct purchase" refers to means for providing an interface such as a link or button that enables a user to make a purchase directly online based on the shopping list they created.

[1520] The system for implementing this invention will grasp the contents of the user's refrigerator and each individual's preferences, suggest recipes based on that, and create a shopping list. It will also have the function of recognizing the user's emotions and adjusting the recipe suggestions according to those emotions. Furthermore, it will send the created shopping list to an online supermarket, providing a path for the user to make direct purchases.

[1521] Hardware and Software Used

[1522] Hardware: Smartphone

[1523] software:

[1524] EmotionEngine: An engine for recognizing user emotions

[1525] RecipeEngine: An engine that suggests recipes based on what's in the fridge and the user's preferences

[1526] ShoppingListGenerator: An engine that generates shopping lists based on suggested recipes

[1527] requests: An HTTP request library for sending shopping lists to online supermarkets.

[1528] System processing overview

[1529] 1. Refrigerator Content Management: The user inputs the contents of the refrigerator through a smartphone application, which allows the system to track the types and quantities of food and beverages in the refrigerator.

[1530] 2. Understanding Preferences: Recording the user’s food preferences and tastes based on the user’s past ordering history and / or directly entered information.

[1531] 3. Emotion Recognition: Analyzes emotions from user input and behavior to understand the emotional state. For example, if a user inputs "I'm tired today," EmotionEngine will recognize "fatigue."

[1532] 4. Recipe Suggestion: Based on the recognized emotions, the contents of the refrigerator, and the user's preferences, RecipeEngine will suggest appropriate recipes. For example, if a user feels tired, it will suggest recipes that have a relaxing effect.

[1533] 5. Shopping list generation: The ingredients and seasonings required for the proposed recipes are listed, and the ShoppingListGenerator generates a shopping list.

[1534] 6. Online supermarket integration: The generated shopping list is sent to an online supermarket, allowing users to directly complete the purchase process based on the list.

[1535] Examples

[1536] When a user types "I'm tired today" into a smartphone application, the Emotion Engine recognizes the "fatigue" and suggests recipes that have a relaxing effect (e.g. herbal tea or light meals). Based on the suggested recipe, a shopping list containing the necessary ingredients is generated and sent to the online supermarket. The user can then directly make a purchase based on the list.

[1537] Examples of prompt statements

[1538] When a user types "I'm tired today," the emotion engine should recognize this and suggest recipes that will help you relax. Then, generate a shopping list based on the suggested recipes and send it to the online supermarket.

[1539] In this way, recipe suggestions and shopping lists can be created according to the user's emotional state, increasing user satisfaction and convenience.

[1540] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1541] Step 1:

[1542] The user inputs the contents of the refrigerator through a smartphone application.

[1543] Input: Type and quantity of food and drinks in the refrigerator

[1544] Output: Refrigerator contents data

[1545] Specific operation: The user enters the ingredients in the refrigerator (e.g., tomatoes, cheese, lettuce) and their quantities in the application's input form, and presses the submit button. The server receives this data and stores it in the database.

[1546] Step 2:

[1547] Record user preferences based on past order history or directly entered information.

[1548] Input: Past order history, direct input of user preferences

[1549] Output: User preference data

[1550] Specific operation: The user enters preferences such as "I like Italian food" or "I'm vegetarian" in the app's settings screen. The server receives this information and stores it in a database.

[1551] Step 3:

[1552] The user inputs the emotion and the server recognizes the emotion.

[1553] Input: Input about the user's emotions (e.g. "I'm tired today")

[1554] Output: Recognized emotion data (e.g. "fatigue")

[1555] Specific operation: The user enters "I'm tired today" into the emotion input screen of the app and presses the send button. The server receives this input and recognizes it as "fatigue" using the Emotion Engine.

[1556] Step 4:

[1557] The server suggests recipes based on what's in the fridge, the user's preferences, and perceived emotions.

[1558] Input: fridge contents data, user preference data, recognized emotion data

[1559] Output: Proposed recipe data

[1560] How it works: The server uses the RecipeEngine to suggest relaxing recipes (e.g. herbal tea or snacks) based on the contents of the fridge (e.g. tomatoes, cheese, lettuce), the user's preferences (e.g. Italian, vegetarian), and perceived emotions (e.g. fatigue).

[1561] Step 5:

[1562] The server generates a shopping list based on the suggested recipes.

[1563] Input: Proposed recipe data

[1564] Output: Shopping list data

[1565] Specific behavior: The server uses a ShoppingListGenerator to generate a shopping list by listing the ingredients and condiments needed for the suggested recipes (e.g. herbs for herbal tea, bread for snacks).

[1566] Step 6:

[1567] The server transmits the generated shopping list to the online supermarket, providing a path for the user to make direct purchases.

[1568] Input: Shopping list data

[1569] Output: Order data for online supermarkets, purchase link

[1570] Specific operation: The server uses the HTTP request library (requests) to send the generated shopping list to the online supermarket's API. In response, the online supermarket returns a purchase link that is displayed to the user. The user can click on this link to proceed with the purchase directly.

[1571] Example 2

[1572] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1573] Conventional recipe suggestion systems have limited recipe suggestions based on the contents of the user's refrigerator and preferences, and have issues with not being able to suggest meals based on the user's emotional state. In addition, there are also issues with creating shopping lists based on suggested recipes and the cumbersome purchasing process at online supermarkets.

[1574] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1575] In this invention, the server includes a means for grasping the contents of the user's refrigerator and each person's preferences, a means for sending a prompt sentence to the generative AI model based on the grasped information and generating a recipe, a means for creating a shopping list based on the generated recipe, a means for sending the created shopping list to the online supermarket, and a means for the user to purchase the ingredients they need at the online supermarket. This makes it possible to suggest recipes based on the contents of the user's refrigerator and preferences, and also to suggest meals according to the user's emotional state. In addition, the process from creating the shopping list to the purchase procedure at the online supermarket can be done in a consistent manner, improving user convenience.

[1576] A "user" is an individual who uses the system to input the contents of their refrigerator and their preferred cooking genres and receives recipe suggestions and shopping lists.

[1577] "What's in the refrigerator" refers to information about ingredients and foods currently stored in the user's refrigerator.

[1578] A "favorite cuisine genre" is a particular style or type of cuisine selected by a user (e.g., Japanese, Western, Chinese, etc.).

[1579] A "generative AI model" is an artificial intelligence algorithm that generates appropriate recipes based on user input information.

[1580] A "prompt" is a text-based instruction sent to a generative AI model to request recipe generation.

[1581] A "recipe" is a list of instructions and ingredients needed to make a particular dish.

[1582] A "shopping list" is a list of ingredients and seasonings needed based on the generated recipe.

[1583] An "online supermarket" is an e-commerce platform where you can purchase food and daily necessities online.

[1584] An "emotional state" is a user's current psychological or emotional state (e.g. feeling stressed, wanting to relax, etc.) that they input into the system.

[1585] This invention is a system that identifies the contents of a user's refrigerator and their preferred cooking genres, uses a generative AI model to suggest recipes based on that information, and also allows them to purchase the necessary ingredients from an online supermarket.

[1586] The server receives information about the contents of the refrigerator and the user's favorite food genres input through the terminal. For example, if the user inputs "eggs and bacon" and selects "Japanese food" as a preference, the server generates a prompt based on this information.

[1587] Examples of specific prompts are as follows:

[1588] I have eggs and bacon in the fridge. Can you suggest some Japanese recipes?

[1589] The server sends this prompt to a generative AI model (e.g., GPT-4). The generative AI model analyzes the prompt and generates an appropriate recipe. For example, a recipe for "Japanese-style scrambled eggs with eggs and bacon" is generated.

[1590] Next, the server creates a shopping list based on the recipe, listing the condiments and vegetables needed to accompany the meal. For example, the following shopping list is created:

[1591] soy sauce

[1592] Mirin

[1593] spinach

[1594] The server then sends this shopping list via the API of the online supermarket the user uses, allowing the user to purchase the ingredients they need all at once via the online supermarket's application or website.

[1595] Furthermore, if the user inputs "I'm feeling stressed" into the emotion engine, the server can send an additional prompt to the generative AI model, suggesting a relaxing drink (e.g., "chamomile tea").

[1596] Examples of specific prompts are as follows:

[1597] There are eggs and bacon in the fridge. Can you suggest some Japanese recipes? Also, the user is feeling stressed so can you suggest some drinks that will help them relax?

[1598] In this way, it is possible to suggest recipes based on the contents of the user's refrigerator and preferences, and even meal suggestions based on the user's emotional state. It also makes it possible to create a shopping list and complete the purchase process at the online supermarket in one go, improving user convenience.

[1599] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1600] Step 1:

[1601] The user inputs the contents of the refrigerator and their favorite cuisine genre.

[1602] The user uses an application on a device (smartphone or PC) to input information about ingredients in the refrigerator (e.g. eggs and bacon) and their preferred cuisine (e.g. Japanese food). The input data is sent to the server.

[1603] Input: contents of fridge (eggs, bacon), favorite food genre (Japanese food)

[1604] Output: User input data sent to the server

[1605] Step 2:

[1606] The server receives the input data and sends prompts to the generative AI model.

[1607] The server receives the ingredient information and cuisine genre sent by the user, then generates a prompt based on this information and sends it to the generative AI model.

[1608] Specific prompt examples:

[1609] I have eggs and bacon in the fridge. Can you suggest some Japanese recipes?

[1610] Input: User input data (eggs, bacon, Japanese food)

[1611] Output: The prompt sent to the generative AI model

[1612] Step 3:

[1613] The generative AI model generates a recipe and returns it to the server.

[1614] The generative AI model receives the prompt and generates an appropriate recipe, for example, "Japanese-style scrambled eggs with eggs and bacon." The generated recipe is returned to the server.

[1615] Input: Prompt statement

[1616] Output: Generated recipe (Japanese-style scrambled eggs with eggs and bacon)

[1617] Step 4:

[1618] The server creates a shopping list based on the recipe.

[1619] The server analyzes the recipe returned by the generative AI model and lists the necessary seasonings, side dishes, etc. This creates a shopping list.

[1620] A concrete shopping list example:

[1621] soy sauce

[1622] Mirin

[1623] spinach

[1624] Input: Generated recipe

[1625] Output: Shopping list

[1626] Step 5:

[1627] The server sends the shopping list to the online supermarket.

[1628] The server sends the created shopping list via the API of the online supermarket the user uses, allowing the user to easily purchase the ingredients they need from the online supermarket.

[1629] Input: Shopping list

[1630] Output: Shopping list sent to online supermarket

[1631] Step 6:

[1632] The user purchases the necessary ingredients from an online supermarket.

[1633] Users can check the shopping list sent from the server via the online supermarket's application or website and purchase the ingredients they need all at once.

[1634] Input: Shopping list sent to online supermarket

[1635] Output: Purchased ingredients

[1636] (Application example 2)

[1637] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1638] Conventional recipe suggestion systems can suggest recipes based on the contents of the user's refrigerator and preferences, but they have a problem in that they cannot suggest recipes that take into account the user's emotional state. In addition, there is no mechanism for easily purchasing the ingredients needed for the suggested recipes, which results in low user convenience.

[1639] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1640] In this invention, the server includes a means for grasping the contents of the user's refrigerator and each user's preferences, a means for suggesting recipes based on the grasped information, a means for creating a shopping list according to the suggested recipes, a means for grasping the user's emotional state, a means for suggesting drinks with a relaxing effect according to the emotional state, and a means for transmitting the created shopping list to an online supermarket and providing a path for the user to make direct purchases. This enables recipe suggestions that take the user's emotional state into account and easy purchase of the necessary ingredients.

[1641] "User" means an individual or organization that uses the System.

[1642] "Contents of the refrigerator" refers to the food and beverages stored in the user's refrigerator.

[1643] "Preferences" refers to the preferences or tastes a user has for particular ingredients or dishes.

[1644] A "recipe" is a list of instructions and ingredients for making a particular dish.

[1645] A "shopping list" is a list of ingredients, seasonings, etc. that a user plans to purchase.

[1646] An "emotional state" is the emotion or mood a user is feeling at a particular point in time.

[1647] A "relaxing drink" is a drink that has the effect of reducing stress and relaxing the user.

[1648] An "online supermarket" is an online store that sells food and daily necessities via the Internet.

[1649] A "path" is the procedure or flow that a user follows to perform a specific operation or action through a system.

[1650] A system for implementing this invention includes means for grasping the contents of a user's refrigerator and each user's preferences, means for suggesting recipes based on the grasped information, means for creating a shopping list based on the suggested recipes, means for grasping the user's emotional state, means for suggesting drinks that have a relaxing effect according to the emotional state, and means for transmitting the created shopping list to an online supermarket and providing a path for the user to make direct purchases.

[1651] Hardware and Software Configuration

[1652] Hardware:

[1653] Smartphone

[1654] server

[1655] software:

[1656] Python

[1657] The Requests library (for sending HTTP requests)

[1658] Database management system (e.g. MySQL)

[1659] Emotion recognition engine (e.g. IBM Watson)

[1660] Data processing and calculation

[1661] Understanding what's in your fridge and what you like:

[1662] Through a smartphone application, users input the ingredients they have in their fridge and their preferences, and this data is sent to a server and stored in a database.

[1663] Recipe suggestions:

[1664] The server uses a generative AI model to suggest appropriate recipes based on the stored ingredients and preference data. For example, if a user inputs "eggs" and "bacon" and selects "Japanese food" as their preference, the system will suggest a recipe for "Japanese-style scrambled eggs with eggs and bacon."

[1665] Emotional state recognition and relaxing drink suggestions:

[1666] Users input their current emotional state through the emotion recognition engine. For example, if they input that they are "stressed," the system will suggest "chamomile tea," which has a relaxing effect.

[1667] Create and send a shopping list:

[1668] Based on the suggested recipe, a shopping list is automatically generated, including the necessary condiments and accompanying vegetables, and this list is sent through the online supermarket's API for easy purchase by the user.

[1669] Examples

[1670] If a user inputs that they have "eggs" and "bacon" in the fridge, selects "Japanese food" as their preference, and inputs "feeling stressed" as their emotional state, the system will suggest a recipe for "Japanese-style scrambled eggs with eggs and bacon" and add "chamomile tea" to the shopping list.

[1671] Example prompt for a generative AI model:

[1672] If a user inputs that they have eggs and bacon in the fridge, selects Japanese food as their preference, and indicates that they are stressed as their emotional state, suggest suitable recipes, add the necessary condiments and vegetable garnishes to their shopping list, and also suggest drinks that will have a relaxing effect.

[1673] In this way, personalized meal suggestions and shopping lists can be generated according to the user's needs.

[1674] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1675] Step 1:

[1676] The user launches a smartphone application and enters the ingredients they have in their fridge and their personal preferences.

[1677] Input: Ingredients (e.g. eggs, bacon), Preferences (e.g. Japanese food)

[1678] Output: The input data is sent to the server.

[1679] Specific behavior: The user enters ingredients and preferences into the application's input form, presses the "Submit" button, and the data is sent to the server.

[1680] Step 2:

[1681] The server stores the received data in a database.

[1682] Input: User-submitted ingredient and preference data

[1683] Output: Ingredients and preferences stored in a database

[1684] Specific operation: The server analyzes the received data and stores it in the corresponding table in the database.

[1685] Step 3:

[1686] Based on the data stored on the server, the server uses a generative AI model to suggest appropriate recipes.

[1687] Input: Ingredients and preferences stored in a database

[1688] Output: Suggested recipe (e.g. Japanese-style scrambled eggs with bacon)

[1689] Specific operation: The server inputs the prompt sentence into the generative AI model, generates an appropriate recipe, and returns the generated recipe to the user.

[1690] Step 4:

[1691] The user inputs their current emotional state through an emotion recognition engine.

[1692] Input: Emotional state (e.g. feeling stressed)

[1693] Output: The input data is sent to the server.

[1694] Specific behavior: When a user enters their emotional state into the application's emotion input form and presses the "Submit" button, the data is sent to the server.

[1695] Step 5:

[1696] The server suggests relaxing drinks based on the emotional state received.

[1697] Input: Emotional state data submitted by the user

[1698] Output: Suggested drink (e.g. chamomile tea)

[1699] What it does: The server analyzes your emotional state and suggests relaxing drinks.

[1700] Step 6:

[1701] The server generates a shopping list based on the suggested recipe and drink, including the necessary condiments and vegetable side dishes.

[1702] Input: Suggested recipe and drink

[1703] Output: Shopping list (e.g. soy sauce, mirin, green onion, chamomile tea)

[1704] What happens: The server parses the proposed recipe and drink and lists the required ingredients.

[1705] Step 7:

[1706] The server sends the generated shopping list to the online supermarket's API, allowing the user to make purchases directly.

[1707] Input: Shopping list

[1708] Output: Order status to online supermarket

[1709] Specific behavior: The server sends the shopping list to the online supermarket's API endpoint and checks whether the order was processed successfully.

[1710] In this way, personalized meal suggestions and shopping lists can be generated according to the user's needs.

[1711] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1712] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image are also input.

[1713] The data generation model 58 performs inference on the input inference data according to the instructions given by the prompts, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1714] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1715] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1716] [Fourth embodiment]

[1717] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1718] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1719] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1720] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. In addition, the microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1721] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1722] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1723] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1724] The control target 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, legs, etc. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, legs, etc. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1725] Fig. 8 shows an example of main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1726] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1727] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1728] In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1729] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1730] "Example 1"

[1731] In one embodiment of the present invention, a user inputs the contents of their refrigerator into a smartphone application. The user's preferences are also understood from their past recipe selection history and from direct input. Based on this information, the system suggests recipes to the user. Based on the suggested recipes, the system automatically creates a shopping list. The created shopping list is sent to the online supermarket that the user uses, and the user can make purchases directly based on the list.

[1732] "Example 2"

[1733] As a specific example, if a user inputs that they have eggs and bacon in their fridge and selects "Japanese food" as their preference, the system will suggest a recipe for "Japanese-style scrambled eggs with eggs and bacon." The system will then add the seasonings and accompanying vegetables needed for that recipe to a shopping list, which will be sent to the online supermarket the user uses. The user can then use that list to purchase the ingredients they need all at once.

[1734] The process flow of each embodiment will be described below.

[1735] "Example 1"

[1736] Step 1: The user opens the application on their smartphone and enters the contents of their refrigerator.

[1737] Step 2: The user inputs their preferences (e.g. Japanese, Western, Chinese, etc.) into the application.

[1738] Step 3: The system suggests recipes based on what's in the user's fridge and their preferences.

[1739] Step 4: The system automatically creates a shopping list based on the suggested recipes.

[1740] Step 5: The system sends the shopping list created by the user to the online supermarket that the user uses.

[1741] Step 6: The user purchases ingredients from the online supermarket based on the shopping list.

[1742] "Example 2"

[1743] Step 1: The user tells the application that they have eggs and bacon in the fridge.

[1744] Step 2: The user enters "Japanese food" as a preference into the application.

[1745] Step 3: The system suggests a recipe for "Japanese-style scrambled eggs with eggs and bacon."

[1746] Step 4: The system adds the condiments and vegetables needed for the recipe to a shopping list.

[1747] Step 5: The system sends the shopping list created by the user to the online supermarket that the user uses.

[1748] Step 6: The user purchases ingredients from the online supermarket based on the shopping list.

[1749] Example 1

[1750] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1751] In the conventional system, the process of users managing the contents of their refrigerator, finding recipes that matched their tastes, and purchasing the necessary ingredients was complicated. In addition, users would create shopping lists without considering the ingredients they already had, which could lead to unnecessary purchases. Furthermore, the purchasing process at online supermarkets was done manually, which was time-consuming and cumbersome.

[1752] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1753] In this invention, the server includes a means for a user to input the contents of the refrigerator, a means for grasping the user's preferences, a means for proposing recipes using a generative AI model based on the grasped information, a means for automatically creating a shopping list based on the proposed recipes, a means for transmitting the created shopping list to an online supermarket, and a means for providing a path for the user to directly purchase at the online supermarket. This enables the user to efficiently manage the contents of the refrigerator, easily find recipes that suit their preferences, and purchase the necessary ingredients without waste.

[1754] "Means for users to input the contents of their refrigerator" refers to an interface that allows users to input the ingredients in their refrigerator using a smartphone or other device.

[1755] "Means of understanding user preferences" refers to a function for collecting and storing a user's food preferences based on the user's past recipe selection history or direct input.

[1756] "Means of suggesting recipes using a generative AI model" refers to a function that uses a generative AI model to generate and suggest appropriate recipes based on data on the contents of the user's refrigerator and their preferences.

[1757] The "means for automatically generating a shopping list" is a function for automatically generating a list of ingredients needed based on a suggested recipe.

[1758] "Means for sending a shopping list to an online supermarket" is a function for sending the created shopping list to the online supermarket used by the user.

[1759] "Means of providing a path for users to make purchases directly from online supermarkets" refers to a function that enables users to purchase groceries directly on the online supermarket platform based on the shopping list submitted to them.

[1760] The present invention is a system that allows a user to manage the contents of a refrigerator, suggest recipes that suit the user's preferences, and purchase the necessary ingredients efficiently. A specific embodiment of this system will be described below.

[1761] System Configuration

[1762] Hardware and Software

[1763] User device: Mobile device such as smartphone or tablet, using applications that run on iOS and Android.

[1764] Server: Located on the cloud, it hosts the database and generative AI model. It uses MySQL for the database and OpenAI's GPT-4 for the generative AI model.

[1765] Entering and Saving Data

[1766] The user enters the contents of the refrigerator

[1767] A user opens the smartphone application and inputs the ingredients in the refrigerator, either manually, by voice, by scanning a barcode, etc. For example, a user inputs "chicken, carrots, onions."

[1768] Understand user preferences

[1769] The user's preferences are captured from their past recipe selection history or through direct input. For example, if a user has previously selected "curry" as their favorite dish, that information is also stored in the database.

[1770] Recipe Suggestions

[1771] Server suggests recipes

[1772] The server uses a generative AI model to suggest recipes based on the contents of the user's fridge and their preferences. Specifically, it sends the following prompt to the generative AI model:

[1773] Contents of user's fridge: Chicken, carrots, onions

[1774] User preference: Curry

[1775] Generate suggested recipes.

[1776] The generative AI model generates a recipe for "chicken curry" based on this prompt, and the server sends the recipe to the user's smartphone.

[1777] Create and send a shopping list

[1778] The server creates the shopping list.

[1779] When a user selects a suggested recipe, the server automatically creates a shopping list of ingredients needed for that recipe, for example, "chicken, carrots, onions, curry roux, and any other necessary seasonings."

[1780] The server sends the shopping list to the online supermarket.

[1781] The server then sends the created shopping list to the online supermarket the user uses, using an API to link with the online supermarket's system.

[1782] Making a purchase

[1783] The user purchases from the online supermarket

[1784] The user opens the online supermarket's application or website, checks the shopping list sent from the server, and can then directly purchase the ingredients he or she needs based on the list.

[1785] Examples

[1786] For example, if a user inputs that they have "chicken, carrots, and onions" in their fridge and has previously preferred "curry," the system will suggest a recipe for "chicken curry." Here is an example of a prompt to be input to the generative AI model:

[1787] Contents of user's fridge: Chicken, carrots, onions

[1788] User preference: Curry

[1789] Generate suggested recipes.

[1790] Based on this prompt, the generative AI model generates a recipe for "chicken curry" and suggests it to the user. Based on the suggested recipe, the system creates a shopping list of "chicken, carrots, onions, curry roux, and other necessary seasonings" and sends it to the online supermarket. The user can use this list to directly purchase the necessary ingredients.

[1791] In this way, users can efficiently manage the contents of their refrigerator, easily find recipes that suit their tastes, and buy the ingredients they need without waste.

[1792] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1793] Step 1:

[1794] The user enters the contents of the refrigerator

[1795] The user opens the smartphone application and inputs the ingredients in the refrigerator. Input methods include manual input, voice input, barcode scanning, etc. For example, the user inputs "chicken, carrots, onions." This input data is stored in a database on the cloud. The input data includes the names of ingredients and their quantities.

[1796] Step 2:

[1797] Understand user preferences

[1798] A user's preferences are understood from past recipe selection history or direct input. For example, if a user has previously selected "curry" as their favorite food, that information is also stored in the database. The server analyzes the user's selection history and extracts preference trends. The input data is the past recipe selection history or direct input of the user's preferences, and the output data is the user's preference trends.

[1799] Step 3:

[1800] Server suggests recipes

[1801] The server uses a generative AI model to suggest recipes based on the contents of the user's fridge and their preferences. Specifically, it sends the following prompt to the generative AI model:

[1802] Contents of user's fridge: Chicken, carrots, onions

[1803] User preference: Curry

[1804] Generate suggested recipes.

[1805] The generative AI model generates a recipe for "chicken curry" based on this prompt, and the server sends the recipe to the user's smartphone. The input data is the contents of the refrigerator and the user's preferences, and the output data is the suggested recipe.

[1806] Step 4:

[1807] The server creates the shopping list.

[1808] When a user selects a suggested recipe, the server automatically creates a shopping list of ingredients needed based on the recipe. For example, it generates a list of "chicken, carrots, onions, curry roux, and other necessary seasonings." The server also provides a function to remove ingredients from the list that the user already has. The input data is the selected recipe and the contents of the refrigerator, and the output data is the shopping list.

[1809] Step 5:

[1810] The server sends the shopping list to the online supermarket.

[1811] The server sends the created shopping list to the online supermarket that the user uses. An API is used to send the list, linking with the online supermarket's system. The input data is the shopping list and the online supermarket's API information, and the output data is the shopping list sent to the online supermarket.

[1812] Step 6:

[1813] The user purchases from the online supermarket

[1814] The user opens the online supermarket application or website and checks the shopping list sent from the server. The user can directly purchase the ingredients he needs based on the list. The input data is the shopping list sent to the online supermarket, and the output data is the purchased ingredients.

[1815] (Application example 1)

[1816] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1817] Conventional refrigerator management systems had the problem that users had to manually manage the contents of their refrigerators, search for recipes themselves, and list the ingredients they needed to purchase, which was a lot of work. In addition, it was difficult to suggest recipes that matched the user's preferences, and creating shopping lists and purchasing procedures were cumbersome. This reduced user convenience and made efficient ingredient management difficult.

[1818] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1819] In this invention, the server includes a means for grasping the contents of the user's refrigerator and each user's preferences, a means for proposing recipes based on the grasped information, a means for creating a shopping list according to the proposed recipes, and a means for transmitting the created shopping list to an electronic commerce platform and providing a path for the user to make direct purchases. This enables the user to efficiently manage the contents of the refrigerator, receive recipe suggestions according to their preferences, and easily purchase the necessary ingredients.

[1820] A "user" is an individual who uses the system to input the contents of their refrigerator and their preferences and receive recipe suggestions and shopping lists.

[1821] "What's in the refrigerator" refers to information about the food and beverages currently stored in the user's refrigerator.

[1822] "Preferences" are the ingredients and cooking preferences that a user indicates based on their past recipe selection history or direct input.

[1823] A "recipe" is a list of instructions and ingredients needed to make a particular dish.

[1824] A "shopping list" is a list of ingredients and products to purchase that is created based on a suggested recipe.

[1825] An "e-commerce platform" is an online marketplace for purchasing goods and services via the Internet.

[1826] A "guideline" is the sequence of steps or operations that allow a user to directly purchase items based on their shopping list.

[1827] A "generative AI model" is an artificial intelligence algorithm used to suggest optimal recipes based on what's in a user's fridge and their preferences.

[1828] A "prompt sentence" is an instruction sentence for suggesting recipes or creating a shopping list that is generated by a generative AI model based on information input by the user.

[1829] A system for implementing the present invention includes means for identifying the contents of a user's refrigerator and each user's preferences, means for suggesting recipes based on the identified information, means for creating a shopping list based on the suggested recipes, and means for transmitting the created shopping list to an e-commerce platform and providing a path for the user to make direct purchases.

[1830] System Programming

[1831] The server provides an interface for users to input the contents of their refrigerator using their smartphones. The user's preferences are understood from their past recipe selection history and direct input. This information is sent to the server and stored in a database.

[1832] The server uses a generative AI model to suggest optimal recipes based on the contents of the user's refrigerator and their preferences. The generative AI model generates prompts based on the user's input information and suggests recipes.

[1833] Based on the suggested recipes, the server automatically creates a shopping list, which is then sent to the e-commerce platform for users to purchase directly.

[1834] Hardware and Software Used

[1835] Hardware: Smartphones, servers

[1836] Software: Python, Requests library, generative AI models

[1837] Data processing and calculation

[1838] 1. User inputs the contents of the refrigerator: The user inputs the contents of the refrigerator using a smartphone application. This information is sent to the server and stored in a database.

[1839] 2. Understanding user preferences: User preferences are understood from past recipe selection history and direct input. This information is also sent to the server and stored in the database.

[1840] 3. Recipe Suggestion: The server uses a generative AI model to suggest optimal recipes based on the contents of the user's refrigerator and their preferences. The generative AI model generates prompts based on the user's input information and makes recipe suggestions.

[1841] 4. Create shopping list: Based on the suggested recipes, the server automatically creates a shopping list.

[1842] 5. Send shopping list: Once created, the shopping list is sent to the e-commerce platform to allow the user to make purchases directly.

[1843] Examples

[1844] Suppose a user inputs that they have “Ingredient A” and “Ingredient B” in their fridge, and that they prefer “vegetarian” and “non-spicy” food. If they have previously selected “Recipe A” and “Recipe B,” the system will suggest “Recipe C” and “Recipe D.” It will then generate a shopping list that includes “Ingredient 1” and “Ingredient 2” and send it to the e-commerce platform.

[1845] Examples of prompt statements

[1846] User ID: 12345

[1847] Contents of the fridge: ["ingredient A", "ingredient B"]

[1848] User Preferences: {"Vegetarian": True, "Spicy": False}

[1849] Past recipe selection history: ["Recipe A", "Recipe B"]

[1850] Suggested recipes: ["Recipe C", "Recipe D"]

[1851] Generated shopping list: ["ingredient 1", "ingredient 2"]

[1852] Sending status to online supermarket: 200

[1853] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1854] Step 1:

[1855] The user inputs the contents of the refrigerator using a smartphone application.

[1856] Input: Information about ingredients in the user's refrigerator (e.g., ingredient A, ingredient B)

[1857] Data processing: The entered ingredient information is converted into JSON format and sent to the server.

[1858] Output: The contents of the fridge sent to the server.

[1859] Step 2:

[1860] The server knows the user's preferences.

[1861] Input: User's past recipe selection history and direct preference input (e.g. vegetarian, not spicy)

[1862] Data processing: Store user preference information in a database.

[1863] Output: User preferences stored in a database

[1864] Step 3:

[1865] The server suggests recipes using a generative AI model.

[1866] Input: Fridge contents data, user preferences data

[1867] Data processing: A generative AI model uses this data to generate prompts and suggest optimal recipes.

[1868] Output: Suggested recipes (e.g. Recipe C, Recipe D)

[1869] Step 4:

[1870] The server creates a shopping list based on the suggested recipes.

[1871] Input: Suggested recipe data

[1872] Data processing: List ingredients needed for recipes and generate shopping lists.

[1873] Output: The created shopping list (e.g. ingredient 1, ingredient 2)

[1874] Step 5:

[1875] The server transmits the created shopping list to the e-commerce platform.

[1876] Input: The shopping list data created

[1877] Data processing: Formatting the shopping list to send to the e-commerce platform's API.

[1878] Output: Status of submission to e-commerce platform (e.g. 200 OK)

[1879] Step 6:

[1880] A user purchases products based on a shopping list through an e-commerce platform.

[1881] Input: Shopping list submitted to e-commerce platform

[1882] Data processing: The e-commerce platform searches for products based on the shopping list and completes the purchase.

[1883] Output: A confirmation message for the purchase (e.g. Purchase Completed)

[1884] Example 2

[1885] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1886] Conventional recipe suggestion systems would only suggest appropriate recipes when users input the ingredients they had in their fridge and their preferred cooking genre, but lacked the functionality to automatically list the necessary seasonings and accompaniments and send them to the online supermarket. As a result, users had to manually create a shopping list based on the suggested recipes and then go through the purchasing process separately at the online supermarket. This process was time-consuming and inefficient.

[1887] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1888] In this invention, the server includes a means for grasping the contents of a user's refrigerator and each user's preferences, a means for proposing a recipe using a generative AI model based on the grasped information, a means for creating a shopping list including necessary seasonings and accompaniment ingredients based on the proposed recipe, a means for transmitting the created shopping list to an online supermarket, and a means for providing a path for the user to purchase the necessary ingredients in one go through the online supermarket. As a result, the user is automatically suggested appropriate recipes based on the contents of the refrigerator and their preferences, and is able to efficiently purchase the necessary ingredients.

[1889] "User" refers to an individual who uses the system to input the contents of their refrigerator and their preferred cuisine genres.

[1890] "What's in the refrigerator" refers to the food items currently stored in the user's refrigerator.

[1891] "Favorite Cuisine Genre" refers to a particular style or type of cuisine selected by a User.

[1892] A "generative AI model" refers to an artificial intelligence algorithm that generates an appropriate recipe based on input information.

[1893] A "prompt sentence" refers to an input sentence used to request a generative AI model to generate a recipe.

[1894] A "recipe" is a list of instructions or ingredients for making a particular dish.

[1895] "Condiments" refer to ingredients used to adjust the flavor of food.

[1896] "Accompaniment ingredients" refers to additional ingredients served with the main dish.

[1897] A "shopping list" refers to a list of ingredients needed based on a suggested recipe.

[1898] An "online supermarket" refers to a website or application that provides the service of purchasing food and daily necessities online.

[1899] "Path" refers to the steps and flow that users take to purchase the ingredients they need in one go through an online supermarket.

[1900] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1901] This invention is a system that, when a user inputs the ingredients in the refrigerator and the type of cooking they like, suggests appropriate recipes, automatically creates a shopping list including the necessary seasonings and garnishes, and transmits it to an online supermarket. A specific embodiment of this system is described below.

[1902] 1. System Overview

[1903] The system involves a series of processes: identifying the contents of a user's refrigerator and preferred cooking genres, suggesting recipes using a generative AI model, creating a shopping list based on the suggested recipes, and sending it to the online supermarket.

[1904] 2. Hardware and Software Used

[1905] Hardware: The device used by the user (smartphone, tablet, computer, etc.)

[1906] Software: Server, generative AI model (e.g. OpenAI's GPT-4), online supermarket API

[1907] 3. Program Processing

[1908] 1. Receiving User Input

[1909] The user inputs into the device the ingredients they have in their fridge (e.g. eggs and bacon) and their preferred cuisine type (e.g. Japanese food).

[1910] The terminal transmits the input information to the server.

[1911] 2. Recipe suggestions

[1912] The server analyzes the received information on ingredients and cuisine genre.

[1913] The server sends prompts to the generative AI model, which then generates an appropriate recipe.

[1914] The server sends the generated recipe (e.g. "Japanese-style scrambled eggs with eggs and bacon") back to the terminal.

[1915] 3. Generate a shopping list

[1916] The server analyzes the generated recipe and lists the necessary condiments and side dishes.

[1917] The server sends this shopping list to the API of the online supermarket the user is using.

[1918] 4. Purchase procedure

[1919] The user accesses the online supermarket's website through their device and checks the submitted shopping list.

[1920] Users purchase the ingredients they need in bulk.

[1921] 4. Specific Examples

[1922] The user types into the device, "I have eggs and bacon in the fridge," and selects "Japanese food" as their preferred cuisine genre.

[1923] The server uses a generative AI model to suggest a recipe for "Japanese-style scrambled eggs with eggs and bacon."

[1924] The server adds the necessary seasonings (e.g. soy sauce, mirin) and accompanying vegetables (e.g. spinach) to a shopping list and sends it to the online supermarket.

[1925] Users check their shopping list on the online supermarket website and purchase all the ingredients they need in one go.

[1926] 5. Examples of prompts

[1927] I have eggs and bacon in the fridge. Can you suggest some Japanese recipes?

[1928] The above is a specific embodiment for carrying out the present invention. This system automatically suggests suitable recipes to users based on the contents of their refrigerator and their preferences, enabling them to efficiently purchase the ingredients they need.

[1929] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1930] Program processing flow

[1931] Step 1: Receiving User Input

[1932] The user inputs into the device the ingredients they have in their fridge (e.g. eggs and bacon) and their preferred cuisine type (e.g. Japanese food).

[1933] Input: A user types "eggs and bacon" into a terminal application and selects "Japanese food" from a drop-down menu.

[1934] The terminal transmits the input information to the server.

[1935] Output: The data sent from the terminal to the server is ingredient information and cuisine genre information.

[1936] Specific actions: A user opens an application on their smartphone, types "eggs and bacon" into the text box, selects "Japanese food" from the drop-down menu, and presses the "Submit" button.

[1937] Step 2: Suggest a recipe

[1938] The server analyzes the received information on ingredients and cuisine genre.

[1939] Input: Ingredient information and cuisine genre information received by the server from the terminal.

[1940] The server sends prompts to the generative AI model, which then generates an appropriate recipe.

[1941] Data processing: The server sends a prompt to the generative AI model: "There are eggs and bacon in the fridge. Please suggest a Japanese recipe."

[1942] Output: Recipe information returned from the generative AI model.

[1943] The server sends the generated recipe (e.g. "Japanese-style scrambled eggs with eggs and bacon") back to the terminal.

[1944] Specific operation: The server sends a prompt to the generative AI model and displays the returned recipe on the user's device.

[1945] Step 3: Generate a shopping list

[1946] The server analyzes the generated recipe and lists the necessary condiments and side dishes.

[1947] Input: The generated recipe information.

[1948] Data processing: The server extracts the necessary seasonings (e.g. soy sauce, mirin) and accompanying vegetables (e.g. spinach) from the recipe.

[1949] Output: A list of ingredients needed.

[1950] The server sends this shopping list to the API of the online supermarket the user is using.

[1951] Specific behavior: The server sends a request to the online supermarket's API saying, "Please add these items to your shopping list."

[1952] Step 4: Checkout

[1953] The user accesses the online supermarket's website through their device and checks the submitted shopping list.

[1954] Input: The shopping list sent by the server.

[1955] Output: Shopping list displayed on the online supermarket website.

[1956] Users purchase the ingredients they need in bulk.

[1957] Specific behavior: The user opens the online supermarket app, goes to the "Shopping List" section, checks the list sent from the server, and presses the "Purchase" button.

[1958] The above is the specific processing flow of the program of this system.

[1959] (Application example 2)

[1960] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1961] Conventional recipe suggestion systems were insufficient in suggesting recipes that took into account the ingredients in the user's refrigerator and preferences, and had problems with creating a shopping list based on the suggested recipe and the complicated process of purchasing the necessary ingredients. In addition, there was no way for users to easily purchase additional ingredients required for the suggested recipe, which resulted in low convenience. There is a need to solve these problems and improve user convenience.

[1962] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1963] In this invention, the server includes a means for grasping the contents of a user's refrigerator and each user's preferences, a means for proposing recipes using a generative AI model based on the grasped information, a means for creating a shopping list according to the proposed recipes, and a means for transmitting the created shopping list to a food delivery service and providing a path for the user to directly purchase. This allows the user to receive recipe suggestions based on the ingredients in the refrigerator and their preferences, and further allows the user to easily purchase any additional ingredients that are required.

[1964] A "user" is an individual who uses the system to input the contents of their refrigerator and their preferences and receive recipe suggestions and shopping lists.

[1965] "Contents of the refrigerator" refers to ingredients and foods currently stored in the user's refrigerator.

[1966] "Preferences" refers to a user's personal preferences for a particular cuisine genre or seasoning.

[1967] A "generative AI model" is an artificial intelligence model that generates appropriate recipes based on user input information.

[1968] A "recipe" is a set of instructions that lists the steps and ingredients needed to make a particular dish.

[1969] A "shopping list" is a list of ingredients and seasonings needed based on a suggested recipe.

[1970] A "food delivery service" is a service that delivers ingredients and meals ordered online by users to their homes.

[1971] A "guideline" refers to the flow of procedures and operations that allow users to easily purchase the ingredients they need through the system.

[1972] A system for implementing this invention includes means for understanding the contents of a user's refrigerator and their individual preferences, means for suggesting recipes using a generative AI model based on the understood information, means for creating a shopping list based on the suggested recipes, and means for sending the created shopping list to a food delivery service and providing a path for the user to make direct purchases.

[1973] Hardware and Software Configuration

[1974] Hardware: Smartphones, servers

[1975] Software: Python, OpenAI API, HTTP request library (requests)

[1976] Data processing and calculation

[1977] 1. Receiving user input: The user uses their smartphone to input the ingredients they have in their fridge and their preferred cuisine, for example, "eggs, bacon" and "Japanese food."

[1978] 2. Using a generative AI model: The server uses the OpenAI API to generate a recipe based on the user's input. An example of a prompt is as follows:

[1979] Ingredients in the fridge: eggs, bacon

[1980] Favorite food genre: Japanese food

[1981] Suggested recipe:

[1982] 3. Creating a shopping list: The server extracts the additional ingredients required from the generated recipe and creates a shopping list. For example, "soy sauce, mirin, and green onion" are added.

[1983] 4. Sending an order to a food delivery service: The server sends the created shopping list to the food delivery service and provides a path for the user to purchase directly. This allows the user to easily purchase the ingredients they need.

[1984] Examples

[1985] If a user inputs that they have "eggs and bacon" in their fridge and selects "Japanese food" as their preferred cuisine genre, the server will use the generative AI model to suggest a recipe for "Japanese-style scrambled eggs with eggs and bacon." It will then add "soy sauce, mirin, and green onions" as additional ingredients needed to a shopping list and send this list to a food delivery service. The user can easily purchase these ingredients and have them delivered via their smartphone.

[1986] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1987] Step 1:

[1988] The user uses a smartphone to input the ingredients in the refrigerator and the type of cuisine they prefer.

[1989] Input: Ingredients in the fridge (e.g., "eggs, bacon"), favorite cuisine (e.g., "Japanese food")

[1990] Output: User input data

[1991] Specific actions: A user opens the application on their smartphone, enters ingredients in the fridge into a text box, and selects their preferred cuisine from a drop-down menu.

[1992] Step 2:

[1993] The server receives the user input data and generates the recipe using a generative AI model.

[1994] Input: User-entered data (ingredients in the fridge, favorite cuisine genres)

[1995] Output: Suggested recipe

[1996] Specific operation: The server calls the OpenAI API and sends the following prompt sentence to the generative AI model.

[1997] Ingredients in the fridge: eggs, bacon

[1998] Favorite food genre: Japanese food

[1999] Suggested recipe:

[2000] The generative AI model generates a recipe based on this prompt and returns it to the server.

[2001] Step 3:

[2002] The server extracts any additional ingredients required from the generated recipe and creates a shopping list.

[2003] Input: Suggested recipe

[2004] Output: Shopping list

[2005] Specific behavior: The server parses the generated recipe text and extracts any additional ingredients needed (e.g., "soy sauce, mirin, green onion"), organizes these ingredients into a list, and creates a shopping list.

[2006] Step 4:

[2007] The server creates a shopping list and sends it to the food delivery service, providing a path for the user to make a purchase directly.

[2008] Input: Shopping list

[2009] Output: Order data for food delivery service

[2010] What happens: The server calls the food delivery service's API and sends the shopping list as order data, including the user's address information, to ensure that the order is processed successfully.

[2011] Step 5:

[2012] The user receives an order confirmation from the food delivery service via smartphone and purchases the necessary ingredients.

[2013] Input: Order confirmation from food delivery service

[2014] Output: Purchased ingredients

[2015] How it works: The user receives an order confirmation on their smartphone app, waits for the delivery of the ingredients they need, and then collects the purchased ingredients when the delivery arrives.

[2016] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2017] "Example 1"

[2018] The present invention is a system including a means for grasping the contents of a user's refrigerator and each user's preferences, a means for suggesting recipes based on the grasped information, a means for creating a shopping list based on the suggested recipes, and an emotion engine for recognizing the user's emotions. The emotion engine has a function for adjusting the recipe suggestions and the shopping list according to the user's emotions. For example, when the user is feeling stressed, the emotion engine suggests recipes that include ingredients that have a relaxing effect. Also, when the user is feeling happy, the emotion engine suggests special recipes suitable for celebrations. This makes it possible to suggest meals according to the user's emotional state.

[2019] "Example 2"

[2020] As a concrete example, consider the case where a user inputs to the application that they have eggs and bacon in the fridge and selects "Japanese food" as their preference. Furthermore, suppose the user inputs to the emotion engine that they "feel stressed." In this case, the system will suggest a recipe for "Japanese-style scrambled eggs with eggs and bacon" and create a shopping list that adds "chamomile tea," which has a relaxing effect. This makes it possible to suggest meals according to the user's emotional state.

[2021] The process flow of each embodiment will be described below.

[2022] "Example 1"

[2023] Step 1: The user enters the contents of the fridge and each person's preferences into the application.

[2024] Step 2: The system will understand the information you entered and suggest recipes based on it.

[2025] Step 3: The system creates a shopping list with suggested recipes.

[2026] Step 4: The user inputs their current emotional state into the emotion engine.

[2027] Step 5: The emotion engine adapts recipe suggestions and shopping lists based on the user...

Claims

1. a means for inputting the contents of a user's refrigerator; a means for ascertaining the preferences of said user; means for grasping the emotional state of the user; means for suggesting recipes based on the perceived preferences and emotional state of the user; A means for creating a shopping list based on the suggested recipes; and means for transmitting the created shopping list to an online supermarket.

2. a means for inputting the contents of a user's refrigerator; a means for ascertaining the preferences of said user; means for grasping the emotional state of the user; A means for suggesting recipes based on a prompt sentence combining the identified preferences and the identified emotional state of the user; a means for generating a shopping list based on a suggested recipe, including condiments and accompaniments; and means for transmitting the created shopping list to an online supermarket.

3. a means for inputting the contents of a user's refrigerator; a means for ascertaining the preferences of said user; means for grasping the emotional state of the user; means for suggesting a shopping list including relaxing drinks based on the perceived preferences and emotional state of the user; and means for transmitting the created shopping list to a food delivery service.

Images