system

Abstract

We provide the system. [Solution] A terminal means that transmits ingredient information entered by the user to a server, A server means that receives ingredient information transmitted from the terminal means and stores it in a database, The aforementioned server means generates a weekly menu based on the stored ingredient information, A means for listing missing ingredients necessary for the menu generated by the menu generation means, A display means for displaying to the user the list of insufficient ingredients listed by the aforementioned insufficient ingredient listing means and the weekly menu, A system that includes this.

Description

Technical Field

[0005]

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

Means for Solving the Problems

[0005] This invention solves the above problems by the following means: providing a system that includes terminal means for transmitting ingredient information entered by a user to a server, server means for receiving ingredient information transmitted from the terminal means and storing it in a database, menu generation means for generating a weekly menu based on the ingredient information stored by the server means, ingredient deficiency listing means for listing any missing ingredients needed for the menu generated by the menu generation means, and display means for displaying the ingredient deficiency list and the weekly menu listed by the ingredient deficiency listing means to the user, thereby enabling the user to efficiently plan menus and easily identify any missing ingredients.

[0006] A "user" is an individual or household that uses the system to input ingredient information, generate menus, and list any missing ingredients.

[0007] "Ingredient information" refers to data about the names and quantities of ingredients that the user currently possesses.

[0008] A "server" is a central processing unit that receives ingredient information sent from terminals, stores it in a database, and also generates menus and lists any missing ingredients.

[0009] A "terminal" is an electronic device (e.g., smartphone, personal computer, tablet, etc.) used by the user to input ingredient information and send it to the server.

[0010] A "database" is a system used to store and manage ingredient information and recipe information sent by users within a server.

[0011] The "menu generation method" is a server-based function that automatically generates a week's worth of menus by selecting different types of recipes for each day of the week based on stored ingredient information.

[0012] The "missing ingredient list method" is a server-based function that compares all the ingredients required for a generated menu with the ingredients the user currently possesses and lists any missing ingredients.

[0013] "Display means" refers to the terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[0014] A "recipe database" is a database that stores recipe information for various dishes, and it is the target of search and selection by the menu generation method.

[0015] The "weekly meal plan" is a meal plan for seven days, from Monday to Sunday, and is automatically generated based on the ingredient information entered by the user. [Brief explanation of the drawing]

[0016] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] overview

[0038] This invention is a system that automatically generates an optimal menu based on the user's ingredient information, lists any missing ingredients, and provides them to the user. This system enables users to efficiently plan meals and simplify grocery shopping.

[0039] System Configuration

[0040] This system includes the following components:

[0041] 1. Terminal: An electronic device used by the user to input ingredient information and send it to the server.

[0042] 2. Server: A central processing unit that receives ingredient information sent from terminals and stores it in a database.

[0043] 3. Database: A system for managing stored information on ingredients and recipes.

[0044] 4. Menu generation method: A server-based function that automatically generates a week's worth of menus based on stored ingredient information.

[0045] 5. Method for listing missing ingredients: A server-side function that compares all the ingredients required for the generated menu with the ingredients the user currently possesses and lists any missing ingredients.

[0046] 6. Display means: A terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[0047] Program processing

[0048] Enter and submit ingredient information

[0049] The user enters information about the food items currently in their refrigerator into the terminal. The terminal converts this information into a data structure and sends it to the server. For example, if the user has "salmon," "chicken," "cabbage," and "carrots," the terminal will send this data.

[0050] Food ingredient information storage

[0051] The server receives ingredient information sent from the terminal and stores it in the database along with the user's identification information. This allows for centralized management of the ingredient information held by each user.

[0052] Menu generation

[0053] The server selects appropriate recipes from its recipe database based on saved ingredient information. It searches for recipes based on different themes for each day of the week (e.g., fish dishes on Mondays, meat dishes on Tuesdays) and generates a week's worth of meal plans.

[0054] List of missing ingredients

[0055] The server identifies all the ingredients needed for the generated weekly menu and compares them to the ingredients the user currently possesses. Based on the comparison, it lists any missing ingredients.

[0056] Displaying Results

[0057] The terminal displays the user a weekly menu and a list of missing ingredients received from the server. This allows users to plan their meals and makes it easier to buy the necessary ingredients.

[0058] Specific example

[0059] User: Uses a smartphone to input "salmon," "chicken," "cabbage," and "carrots."

[0060] Terminal: Sends the entered ingredient information to the server.

[0061] Server: Receives ingredient information and saves it to the database.

[0062] Server: Generates a weekly menu. For example, recipes like "Grilled Salmon and Vegetables" might be selected for Monday, and "Stir-fried Chicken" for Tuesday.

[0063] Server: Lists the missing ingredients. For example, it might find that "soy sauce" and "chicken broth" are missing.

[0064] Terminal: Displays the generated weekly menu and a list of missing ingredients to the user.

[0065] In this way, the system efficiently supports users in their meal planning and shopping.

[0066] The following describes the processing flow.

[0067] Step 1:

[0068] The user enters the ingredients currently in their refrigerator into the terminal. For example, they might enter "salmon," "chicken," "cabbage," and "carrots."

[0069] Step 2:

[0070] The terminal sends the entered ingredient information to the server. At this time, the ingredient information, along with the user ID, is converted into a data structure such as JSON format.

[0071] Step 3:

[0072] The server receives ingredient information sent from the terminal. The received data is stored in a database using the user ID as the key.

[0073] Step 4:

[0074] The server uses the stored ingredient information from the database to search for appropriate recipes in the recipe database and generate a weekly meal plan. Recipes are selected based on different themes for each day of the week (e.g., fish dishes on Monday, meat dishes on Tuesday).

[0075] Step 5:

[0076] The server identifies all the ingredients needed for the generated weekly menu. This list is constructed based on recipe information stored in the recipe database.

[0077] Step 6:

[0078] The server compares the list of required ingredients with the ingredients the user currently possesses. Based on the comparison results, it lists the ingredients that are missing.

[0079] Step 7:

[0080] The terminal displays the weekly menu and a list of missing ingredients received from the server to the user. The display is in a visually easy-to-understand format. For example, it might show menus such as "Monday: Grilled salmon and vegetables" and "Tuesday: Stir-fried chicken," along with missing ingredients such as "Soy sauce, chicken broth."

[0081] (Example 1)

[0082] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0083] In today's busy lifestyle, it's not easy for users to efficiently plan meals and shop for groceries. Planning daily meals and checking for missing ingredients is time-consuming and laborious, making it a struggle for many. Therefore, there is a need for a system that automatically generates optimal meal plans based on the user's existing ingredient information and lists any missing ingredients.

[0084] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0085] In this invention, the server includes an electronic device means for transmitting ingredient information entered by the user to the server, a central processing unit means for receiving the ingredient information transmitted from the electronic device means and storing it in a data structure, a menu generation function that automatically generates a weekly menu based on the stored ingredient information, a comparison calculation means for listing any missing ingredients required for the menu generated by the menu generation function, and a visual display means for displaying the list of missing ingredients and the weekly menu listed by the comparison calculation means to the user. As a result, the user can efficiently plan a weekly menu, easily identify any missing ingredients, and create a shopping list.

[0086] "Ingredient information" refers to information about ingredients that the user possesses, including details such as the name of the ingredient, quantity, and expiration date.

[0087] "Electronic device means" refers to a device used by the user to input ingredient information and transmit that information to a server, and is an electronic device such as a smartphone or tablet.

[0088] The "central processing unit" is a server that has the function of receiving food ingredient information transmitted from the electronic device and storing it in a database.

[0089] "Data structure" refers to the format and structure of the data that the server uses to store the transmitted ingredient information.

[0090] The "menu generation function" is a server-based feature that automatically generates a week's worth of menus based on saved ingredient information.

[0091] The "comparison calculation means" is a server-based processing function that compares all the ingredients required for the generated menu with the ingredients the user currently possesses, and identifies any missing ingredients.

[0092] "Visual display means" refers to a terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[0093] 1. Overview

[0094] This invention is a system that automatically generates an optimal menu based on the user's available ingredients and lists any missing ingredients, providing them to the user. This system allows users to efficiently plan meals and simplify grocery shopping.

[0095] 2. System Configuration

[0096] This system includes the following components:

[0097] 1. Terminal: An electronic device (e.g., smartphone) used by the user to input ingredient information and send it to the server.

[0098] 2. Server: A central processing unit that receives ingredient information sent from terminals and stores it in a data structure.

[0099] 3. Database: A system for managing stored information on ingredients and recipes.

[0100] 4. Menu generation function: A server-based function that automatically generates a week's worth of menus based on saved ingredient information.

[0101] 5. Comparison calculation means: A function that compares all the ingredients required for the generated menu with the ingredients the user currently possesses and lists the missing ingredients.

[0102] 6. Visual display means: A terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[0103] 3. Enter and submit ingredient information

[0104] The user enters information about the food items in their refrigerator at home into a device such as a smartphone. For example, if the user has "salmon," "chicken," "cabbage," and "carrots," they enter the names of these food items using a dedicated application on their device. The device converts the entered food information into JSON data format and sends it to the server using the HTTPS protocol.

[0105] 4. Preservation of ingredient information

[0106] The server receives JSON data sent from the terminal. The received ingredient information is stored in the database along with the user's identification information. If new ingredient information duplicates existing information, the data is kept up-to-date by updating the quantity and expiration date.

[0107] 5. Menu creation

[0108] The server searches for appropriate recipes from the recipe database based on ingredient information stored in the database. It filters recipes based on themes for each day of the week (e.g., fish dishes on Mondays, meat dishes on Tuesdays) and generates a week's worth of meal plans. This generation process uses a recipe selection algorithm based on a generative AI model.

[0109] 6. List of missing ingredients

[0110] The server creates a list of all necessary ingredients based on the generated weekly menu. It compares this list to the user's current ingredient list and lists any missing ingredients, including the quantity and type of each ingredient. For example, it might list ingredients the user doesn't have, such as soy sauce or chicken broth.

[0111] 7. Displaying the results

[0112] The server sends the generated weekly meal plan and a list of missing ingredients to the device in JSON format. The device parses the received JSON data and displays it visually to the user. Specifically, it displays the weekly meal plan and the list of missing ingredients in the application's UI.

[0113] 8. Specific Examples

[0114] User: Use a smartphone app to input "salmon," "chicken," "cabbage," and "carrots."

[0115] Terminal: Converts the entered information into JSON data and sends it to the server.

[0116] Server: Receives data and saves it to the database.

[0117] Server: Based on the saved data, it selects recipes such as "Grilled Salmon and Vegetables" and "Stir-fried Chicken" from the recipe database and generates a week's worth of meal plans.

[0118] Server: Based on the generated menu, it lists any missing ingredients such as "soy sauce" or "chicken broth".

[0119] Terminal: Displays the generated menu and a list of missing ingredients to the user.

[0120] Examples of input prompts for a generative AI model

[0121] The following is an example of a prompt to input into the generating AI model:

[0122] "Please create a one-week menu using salmon, chicken, cabbage, and carrots. Also, please list any ingredients that are missing from the menu."

[0123] In this way, the system efficiently supports users in their meal planning and shopping.

[0124] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0125] Step 1:

[0126] Enter and submit ingredient information

[0127] Users enter information about the ingredients they have in their refrigerator at home into a dedicated application on their smartphone.

[0128] Input: The user enters "salmon", "chicken", "cabbage", and "carrots".

[0129] The terminal converts the entered ingredient information into JSON data format. This data includes details such as the name of the ingredient, quantity, and expiration date.

[0130] Data processing: Converting ingredient information to JSON data format.

[0131] Output: Converted JSON data.

[0132] The terminal sends the converted JSON data to the server using the HTTPS protocol.

[0133] Function: Generate and send JSON data.

[0134] Step 2:

[0135] Food ingredient information storage

[0136] The server receives JSON data sent from the terminal. This is done along with the user's identification information.

[0137] Input: JSON data received via the HTTPS protocol.

[0138] The server parses the received JSON data and stores information about each ingredient as a separate entry in the database.

[0139] Data processing: Parsing JSON data and saving it to a database.

[0140] Output: Results of saving the analyzed food ingredient information to the database.

[0141] Function: Saves ingredient information.

[0142] Step 3:

[0143] Menu generation

[0144] The server retrieves all the ingredient information the user possesses from the database. This means that it retrieves the necessary data based on the stored ingredient information.

[0145] Input: Ingredient information stored in the database.

[0146] The server filters recipes from the recipe database based on themes for each day of the week and generates a week's worth of meal plans.

[0147] Data processing: Recipe search and filtering based on ingredient information.

[0148] Output: A weekly meal plan has been generated.

[0149] Function: Recipe selection and menu generation using a generative AI model.

[0150] Step 4:

[0151] List of missing ingredients

[0152] The server creates a list of all necessary ingredients based on the generated weekly menu.

[0153] Input: A generated weekly meal plan.

[0154] The server compares the user's current ingredient list with the generated ingredient list and lists any missing ingredients.

[0155] Data processing: Compare the list of ingredients you have with the list of ingredients you need.

[0156] Output: A list of missing ingredients.

[0157] Operation: Lists missing ingredients through comparison operations.

[0158] Step 5:

[0159] Displaying Results

[0160] The server sends the generated weekly meal plan and a list of missing ingredients to the terminal in JSON format.

[0161] Input: JSON data containing a weekly meal plan and a list of missing ingredients.

[0162] The device parses the received JSON data and displays it visually to the user through the application's UI.

[0163] Data processing: Parsing JSON data and converting it to a display format.

[0164] Output: Weekly meal plan and list of missing ingredients in a displayable format.

[0165] Users can view menus and lists of missing ingredients through their device screen to plan meals in an organized manner.

[0166] Function: Visual display and user confirmation of analyzed data.

[0167] (Application Example 1)

[0168] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0169] Traditional food information management systems generate menus based on the user's existing food information and list missing ingredients. However, they cannot notify users of missing ingredients in real time while shopping, making efficient shopping difficult. As a result, users may overlook necessary ingredients when shopping in physical stores. Furthermore, many systems lack features to guide users to the location of desired ingredients, contributing to prolonged shopping times. Consequently, planned shopping becomes difficult, leading to the problem of unnecessary purchases.

[0170] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0171] In this invention, the server includes terminal means for transmitting ingredient information entered by the user to the server, server means for receiving ingredient information transmitted from the terminal means and storing it in a database, menu generation means for generating a weekly menu based on the stored ingredient information, ingredient shortage list means for listing any missing ingredients needed for the menu generated by the menu generation means, display means for displaying the ingredient shortage list and the weekly menu listed by the ingredient shortage list means to the user, and real-time shopping support means for notifying the user of any missing ingredients in real time when they visit a store and supporting their purchase. As a result, the user can grasp in real time any missing ingredients while shopping at a physical store, enabling them to shop efficiently without missing any necessary ingredients.

[0172] A "terminal device" is an electronic device used to transmit ingredient information entered by the user to a server.

[0173] The "server means" is a central processing unit that receives ingredient information transmitted from the terminal means and stores it in a database.

[0174] The "menu generation means" is a function that automatically generates a weekly menu based on the stored ingredient information of the server means.

[0175] The "means for listing missing ingredients" is a function that lists any missing ingredients needed for the menu generated by the menu generation means.

[0176] "Display means" refers to a device for visually displaying to the user the list of insufficient ingredients and the weekly menu, which have been listed by the insufficient ingredient listing means.

[0177] A "real-time shopping support system" is a function that notifies users in real time of any missing ingredients when they visit a store, and supports their purchase.

[0178] A "database" is a collection of data used to manage information about ingredients and recipes.

[0179] "Comparing" means comparing the ingredients the user currently has with all the ingredients needed for the weekly meal plan to identify any missing ingredients.

[0180] overview

[0181] This invention is a system that automatically generates an optimal menu based on the user's ingredient information, lists any missing ingredients, and provides them to the user. This system enables users to efficiently plan meals and shop for groceries easily. Furthermore, real-time shopping support in stores allows users to purchase necessary ingredients efficiently without missing any.

[0182] System Configuration

[0183] This system includes the following components:

[0184] 1. Terminal device: An electronic device used by the user to input ingredient information and send it to the server. A smartphone is a specific example.

[0185] 2. Server: A central processing unit that receives ingredient information transmitted from terminals and stores it in a database.

[0186] 3. Database: A system for managing stored ingredient information and recipe information. Database management systems such as SQLite are used.

[0187] 4. Menu Generation Method: This is a server-based function that automatically generates a week's worth of menus based on stored ingredient information. It uses a generation AI model that takes into account user preferences and allergy information.

[0188] 5. Method for listing missing ingredients: This is a server-side function that compares all the ingredients required for the generated menu with the ingredients the user currently possesses and lists any missing ingredients.

[0189] 6. Display means: A terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[0190] 7. Real-time shopping support: This function notifies users in real time of any missing ingredients when they visit a store, and supports their purchase.

[0191] Program processing

[0192] Enter and submit ingredient information

[0193] The user uses their smartphone to input information about the food items currently in their refrigerator. This includes barcode scanning and manual entry. The entered food information is converted into a data structure and sent to the server.

[0194] Food ingredient information storage

[0195] The server receives ingredient information sent from the terminal and stores it in the database along with the user's identification information. This allows for centralized management of the ingredient information held by each user.

[0196] Menu generation

[0197] The server selects appropriate recipes from its recipe database based on stored ingredient information. It searches for recipes based on different themes for each day of the week (e.g., fish dishes on Mondays, meat dishes on Tuesdays) and generates a week's worth of meal plans. A generation AI model is used to take user preferences and allergy information into consideration.

[0198] List of missing ingredients

[0199] The server identifies all the ingredients needed for the generated weekly menu and compares them to the ingredients the user currently possesses. Based on the comparison, it lists any missing ingredients.

[0200] Displaying Results

[0201] The terminal displays the user a weekly menu and a list of missing ingredients received from the server. This allows users to plan their meals and makes it easier to buy the necessary ingredients.

[0202] Real-time shopping support

[0203] When users visit a store, they can use their smartphones to check in real time which ingredients are running low. Furthermore, the system can use in-store location information to guide them to the shelf where their desired ingredients are located.

[0204] Specific example

[0205] User: Uses a smartphone to input "chicken," "grilled salmon," "cabbage," and "carrots."

[0206] Server: Receives ingredient information and saves it to the database.

[0207] Server: Uses a generation AI model to generate a weekly menu, such as "Grilled salmon and vegetables" for Monday and "Stir-fried chicken" for Tuesday.

[0208] Server: Lists the missing ingredients. For example, it might find that "onions" and "soy sauce" are missing.

[0209] Display method: The generated weekly menu and a list of missing ingredients are displayed to the user.

[0210] Real-time shopping support: Displays in real time which ingredients are missing at the store the user is visiting, and guides the user to the shelf location using location information.

[0211] Example of a prompt

[0212] "Please list the ingredients the user currently possesses. Next, list all the ingredients needed for the generated weekly meal plan, and then compare and list any missing ingredients."

[0213] In this way, the system efficiently supports users in their meal planning and shopping.

[0214] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0215] Step 1: Enter and submit ingredient information.

[0216] Users input information about ingredients they have at home using their smartphones. Input methods include barcode scanning and manual entry. Specifically, users can scan the barcodes on ingredients using their smartphone's camera or enter text within the app. The entered information is converted into a data structure and sent to the server in JSON format. <Input: Ingredient information (e.g., chicken, cabbage) / Output: Ingredient data in JSON format>

[0217] Step 2: Save ingredient information

[0218] The server receives ingredient information sent from the terminal and stores it in the database along with the user's identification information. The database used here is SQLite. Specifically, the server parses the received JSON data, converts each ingredient's information into the appropriate format, and inserts it into the database. <Input: Ingredient data in JSON format / Output: Ingredient information stored in the database>

[0219] Step 3: Menu Generation

[0220] The server selects appropriate recipes from a recipe database based on ingredient information stored in the database. It uses a generative AI model to generate a week's worth of menus based on a daily theme (e.g., fish dishes on Mondays, meat dishes on Tuesdays). Specifically, the server selects the most suitable recipes considering the user's preferences and allergy information to create a weekly menu. <Input: Ingredient information from the database / Output: Weekly menu list>

[0221] Step 4: List the missing ingredients

[0222] The server identifies all the ingredients needed for the generated weekly menu and compares them with the ingredients the user currently possesses in the database. Based on this comparison, it lists the missing ingredients. Specifically, it matches the list of ingredients included in the menu with the user's ingredient database and extracts only the missing items. <Input: Weekly menu list, ingredient information from the database / Output: List of missing ingredients>

[0223] Step 5: Displaying the results

[0224] The device visually displays a weekly meal plan and a list of missing ingredients, sent from the server, to the user. Specifically, the smartphone app analyzes the received data, formats it for easy viewing on the user interface, and displays it. <Input: Weekly meal plan, list of missing ingredients / Output: Information displayed on the device screen>

[0225] Step 6: Real-time shopping support

[0226] When a user visits a store, they can use their smartphone to check for missing ingredients in real time. This includes a feature that uses in-store location information to guide them to the location of ingredients. Specifically, it uses GPS and in-store beacons to determine the user's current location and displays the optimal route to the shelves where each ingredient is located. <Input: User location information, missing ingredient list / Output: Real-time ingredient location guidance>

[0227] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0228] overview

[0229] This invention combines a system that automatically generates an optimal menu based on the user's ingredient information and lists any missing ingredients with an emotion engine that recognizes the user's emotions. The emotion engine analyzes the user's emotional state and provides a menu based on that state. This system allows users to efficiently plan meals and enjoy meals that suit their emotions.

[0230] System Configuration

[0231] This system includes the following components:

[0232] 1. Terminal: An electronic device used by the user to input ingredient information and send it to the server.

[0233] 2. Server: A central processing unit that receives ingredient information sent from terminals and stores it in a database.

[0234] 3. Database: A system for managing stored information on ingredients and recipes.

[0235] 4. Menu generation method: A server-based function that automatically generates a week's worth of menus based on saved ingredient information and feedback from an emotion engine.

[0236] 5. Method for listing missing ingredients: A server-side function that compares all the ingredients required for the generated menu with the ingredients the user currently possesses and lists any missing ingredients.

[0237] 6. Display means: A terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[0238] 7. Emotion Engine: A system that analyzes the user's emotions and feeds the results back into the menu generation process.

[0239] Program processing

[0240] Enter and submit ingredient information

[0241] The user enters information about the ingredients currently in their refrigerator into the terminal. For example, they might enter "salmon," "chicken," "cabbage," and "carrots." The terminal converts this information into a data structure and sends it to the server.

[0242] Food ingredient information storage

[0243] The server receives ingredient information sent from the terminal and stores it in a database along with the user's identification information. This allows for centralized management of the ingredient information held by each user.

[0244] User emotion recognition

[0245] While the user is using the device, the emotion engine analyzes the user's input and voice to recognize their current emotional state. For example, it can detect emotions such as "tired," "stressed," or "happy."

[0246] Menu generation

[0247] The server uses ingredient information stored in the database and receives feedback from the emotion engine. It then selects appropriate recipes from the recipe database to generate a weekly meal plan. Recipes are selected based on different themes for each day of the week (e.g., fish dishes on Monday, meat dishes on Tuesday), and dishes that reflect the user's emotional state are suggested.

[0248] List of missing ingredients

[0249] The server identifies all the ingredients needed for the generated weekly menu. This list is constructed based on recipe information stored in the recipe database. The system compares the list of required ingredients with the ingredients the user currently possesses, and lists any missing ingredients.

[0250] Displaying Results

[0251] The device displays the user a weekly menu and a list of missing ingredients received from the server, along with suggestions based on the user's emotional state. For example, it might display menus such as "Monday: Grilled salmon and vegetables to replenish energy" and "Tuesday: Relaxing chicken dish," along with a note saying, "Missing: Soy sauce, chicken broth."

[0252] Specific example

[0253] User: Uses a smartphone to input "salmon," "chicken," "cabbage," and "carrots."

[0254] Terminal: Sends the entered ingredient information to the server.

[0255] Server: Receives ingredient information and saves it to the database.

[0256] Emotion Engine: Recognizes that the user is "tired" from their voice input and feeds that information back to the server.

[0257] Server: Based on feedback from the emotion engine, it selects recipes with a refreshing effect and generates a weekly meal plan. For example, Monday might be a "refreshing salad," Tuesday a "relaxing soup," and so on.

[0258] Server: List the missing ingredients and identify the missing ingredients as "soy sauce" and "chicken broth".

[0259] Terminal: Visually displays the generated weekly menu and a list of missing ingredients to the user.

[0260] This system efficiently plans meals while reflecting the user's emotional state and identifies missing ingredients, thereby enriching the user's life.

[0261] The following describes the processing flow.

[0262] Step 1:

[0263] The user enters the ingredients currently in their refrigerator into the terminal. For example, they might enter "salmon," "chicken," "cabbage," and "carrots."

[0264] Step 2:

[0265] The terminal sends the entered ingredient information to the server. At this time, the ingredient information, along with the user ID, is converted into a data structure such as JSON format.

[0266] Step 3:

[0267] The server receives ingredient information sent from the terminal. The received data is stored in a database using the user ID as the key.

[0268] Step 4:

[0269] While the user is using the device, the emotion engine analyzes the user's input and voice. For example, it can recognize emotions such as "tired," "stressed," or "happy."

[0270] Step 5:

[0271] The emotion engine sends the recognized user's emotional state to the server. For example, the data is sent in the format "User ID: user123, Emotional State: Tired".

[0272] Step 6:

[0273] The server receives feedback from the emotion engine based on stored ingredient information in the database. It then selects appropriate recipes from the recipe database based on different themes for each day of the week, generating a weekly menu. For example, a "refreshing recipe" might be selected for Monday, and a "relaxing recipe" for Tuesday.

[0274] Step 7:

[0275] The server generates a list of all the ingredients needed for the generated week's menu. This is done based on recipe information stored in the recipe database.

[0276] Step 8:

[0277] The server compares the created list of required ingredients with the ingredient information the user possesses. Based on the comparison results, it lists the ingredients that are missing.

[0278] Step 9:

[0279] The terminal displays the user a weekly menu and a list of missing ingredients received from the server. For example, the menu might be displayed in the format of "Monday: Grilled salmon and vegetables to replenish energy" and "Tuesday: Relaxing chicken dish," with "soy sauce" and "chicken broth" being indicated as missing ingredients.

[0280] (Example 2)

[0281] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0282] In today's busy lifestyle, there is a need to efficiently manage ingredients and plan meals, but current systems have difficulty suggesting meals that take into account the user's emotional state. As a result, users may not be able to eat meals that suit their specific emotional state, which could lead to decreased satisfaction with meals. This invention aims to achieve more personalized meal suggestions and necessary ingredient management by taking the user's emotional state into consideration.

[0283] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0284] In this invention, the server includes a terminal means for transmitting food ingredient information input by the user to the server, a server means for receiving the food ingredient information transmitted from the terminal means and storing it in a database, a menu generation means for generating a weekly menu based on the food ingredient information stored by the server means, an emotion analysis means connected to the server means for analyzing the emotional state of the user and providing the feedback to the menu generation means, a shortage ingredient listing means for listing up the shortage ingredients required for the menu generated by the menu generation means, and a display means for displaying the shortage ingredient list and the weekly menu listed up by the shortage ingredient listing means to the user. Thereby, it becomes possible to propose a personalized menu considering the emotional state of the user and list up the shortage ingredients.

[0285] The "terminal means" is an electronic device for transmitting food ingredient information input by the user to the server.

[0286] The "server means" is a central processing unit for receiving the food ingredient information transmitted from the terminal means and storing it in a database.

[0287] The "menu generation means" is a means having a function of generating a weekly menu based on the food ingredient information stored by the server means.

[0288] The "emotion analysis means" is a system connected to the server means for analyzing the emotional state of the user and providing the feedback to the menu generation means.

[0289] The "shortage ingredient listing means" is a means for listing up the shortage ingredients required for the menu generated by the menu generation means.

[0290] The "display means" is a function of the terminal for displaying the shortage ingredient list and the weekly menu listed up by the shortage ingredient listing means to the user.

[0291] Embodiments for Implementing the Invention

[0292] This invention combines a system that automatically generates an optimal menu based on ingredient information entered by the user and lists any missing ingredients with an emotion analysis means that recognizes the user's emotions. The following describes in detail how this system works.

[0293] System Components

[0294] This system includes the following components:

[0295] 1. Terminal device: An electronic device used by the user to input ingredient information and send it to the server. Examples include smartphones, tablets, and personal computers.

[0296] 2. Server: A central processing unit that receives ingredient information and stores it in a database. The server uses an SQL database to store ingredient information.

[0297] 3. Database: A system for managing stored ingredient information and recipe information. Examples include database management systems such as MySQL (registered trademark) and PostgreSQL.

[0298] 4. Menu generation method: A server-based function that automatically generates a week's worth of menus based on stored ingredient information and feedback from an emotion analysis method.

[0299] 5. Emotion Analysis System: A system that analyzes the user's emotional state and provides the results to the menu generation system. Natural language processing technology is used for emotion analysis.

[0300] 6. Method for listing missing ingredients: A server-side function that compares all the ingredients required for the generated menu with the ingredients the user currently possesses and lists any missing ingredients.

[0301] 7. Display method: A terminal function that visually displays the generated weekly menu and list of missing ingredients to the user. This may be done using a smartphone app or a web browser.

[0302] System operation

[0303] 1. The user inputs the ingredient information in the refrigerator into the terminal. Specifically, open the app on the smartphone and input ingredients such as "salmon", "chicken", "cabbage", "carrot", etc.

[0304] 2. The terminal converts these input information into a data structure (for example, JSON format) and sends it to the server. The transmission uses an HTTP POST request.

[0305] 3. The server analyzes the received data and saves it in the database together with the user's identification information. In this way, the ingredient information for each user is centrally managed.

[0306] 4. When the user inputs an emotional state such as "I'm tired today" by voice, the terminal analyzes this voice and sends it to the emotion analysis means. The emotion analysis means analyzes this information using natural language processing technology and feedbacks an emotional state such as "tired".

[0307] 5. Based on the ingredient information and the feedback from the emotion analysis means, the server selects an appropriate recipe from the recipe database and generates a weekly menu. The menu is composed of different themes for each day of the week, and dishes based on the user's emotional state are proposed.

[0308] 6. The server lists all the ingredients required for the weekly menu and compares it with the ingredient information the user currently has. The missing parts are listed up, and "soy sauce", "chicken broth", etc. are identified.

[0309] 7. The terminal visually displays the generated weekly menu and the list of missing ingredients. For example, it is displayed as "Monday: Grilled salmon and vegetables to replenish energy", "Tuesday: Chicken dishes to relax", etc.

[0310] Specific examples and example prompt sentences

[0311] Example: A user inputs "salmon," "chicken," "cabbage," and "carrots" from their refrigerator using their smartphone, and enters "tired" as their current emotional state. The system then selects recipes with a refreshing effect and generates a week's worth of meal plans.

[0312] Example of a prompt:

[0313] "I have salmon, chicken, cabbage, and carrots in my refrigerator. I'm also very tired right now. Based on that, please suggest a meal plan for the week and tell me what ingredients I'm missing."

[0314] This system allows users to easily create optimal menus that reflect their emotional state and automatically identify any missing ingredients.

[0315] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0316] Step 1:

[0317] The user enters food information on their device. The user opens the app on their smartphone and enters the food items they have in their refrigerator, such as "salmon," "chicken," "cabbage," and "carrots."

[0318] Input: The name of the ingredient entered by the user.

[0319] Output: The input data is saved to the terminal.

[0320] Specific action: The user enters the name of the ingredient into a text input field in the smartphone app and presses the "Submit" button.

[0321] Step 2:

[0322] The terminal converts the entered ingredient information into a data structure and sends it to the server. The terminal converts the entered text information into JSON format and sends an HTTP POST request to the server.

[0323] Input: The name of the ingredient entered by the user.

[0324] Output: Ingredient information sent to the server (in JSON format).

[0325] Specific operation: The program converts the ingredient information entered on the terminal into JSON format and sends an HTTP POST request to the URL endpoint.

[0326] Step 3:

[0327] The server receives the submitted ingredient information and stores it in the database. The server receives an HTTP request, parses the JSON data, and stores it in the SQL database.

[0328] Input: Ingredient information (in JSON format) sent from the device.

[0329] Output: Ingredient information stored in the database.

[0330] Specific operation: The server receives ingredient information using a REST API, parses it, and executes an INSERT query to insert it into the database along with the user's identification information, and then saves it.

[0331] Step 4:

[0332] The user enters their emotional state. They use their device to voice-input "I'm tired today."

[0333] Input: User's emotional state.

[0334] Output: The input data is saved to the terminal.

[0335] Specific operation: The user uses the voice input function to communicate their emotional state to the device.

[0336] Step 5:

[0337] The device transmits its emotional state to the emotion analysis system. The emotion analysis system uses natural language processing to analyze this information and provides feedback on the emotional state, such as "fatigue."

[0338] Input: User's emotional state.

[0339] Output: Emotional data analyzed by the emotion analysis tool.

[0340] Specific operation: The device converts the audio data into text data, which is then sent to the sentiment analysis engine. The analysis engine uses a natural language processing model to identify the emotional state (e.g., tired) and returns the result.

[0341] Step 6:

[0342] The server generates a menu based on ingredient information and emotional feedback from emotion analysis tools. The server retrieves ingredient information from a database, selects appropriate recipes based on the feedback, and generates a weekly menu.

[0343] Input: Food ingredient information obtained from a database, emotional feedback from emotion analysis tools.

[0344] Output: A week's worth of menus.

[0345] Specific operation: The server retrieves stored ingredient information using SQL queries, selects a suitable recipe from the recipe database considering the sentiment analysis results, and creates a weekly meal plan. For example, for a "tired" state, it selects a recipe that provides energy.

[0346] Step 7:

[0347] The server generates a weekly menu and creates a list of necessary ingredients. It then compares this list with the user's existing ingredient information and lists any missing ingredients.

[0348] Input: Weekly menu, user ingredient information stored in the database.

[0349] Output: List of missing ingredients.

[0350] Specific operation: The server identifies all ingredients included in the week's menu, compares them with the current user's ingredient information using an SQL query, and lists any missing ingredients.

[0351] Step 8:

[0352] The terminal displays the generated weekly menu and a list of missing ingredients to the user. It receives response data from the server and displays it on the user interface.

[0353] Input: Weekly menu and list of missing ingredients from the server.

[0354] Output: Weekly meal plan and list of missing ingredients displayed to the user.

[0355] Specific operation: The device receives a response from the server and displays menus such as "Monday: Grilled salmon and vegetables to replenish energy" and "Tuesday: Relaxing chicken dish" through the application's UI, as well as a list of missing ingredients such as "Missing: Soy sauce, chicken broth".

[0356] (Application Example 2)

[0357] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0358] In modern households, food management and meal planning are not done efficiently, making it particularly difficult to suggest appropriate meals that suit the user's emotional state. Furthermore, the inability to select the optimal ingredients leads to food waste.

[0359] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes terminal means for transmitting ingredient information entered by the user to the server, server means for receiving the ingredient information transmitted from the terminal means and storing it in a database, menu generation means for generating a weekly menu based on the ingredient information stored by the server means, ingredient deficiency listing means for listing any missing ingredients needed for the menu generated by the menu generation means, display means for displaying the ingredient deficiency list and the weekly menu listed by the ingredient deficiency listing means to the user, emotion recognition means for recognizing the user's emotional state using an emotion engine provided in the server means, suggestion means for making suggestions based on the user's emotional state recognized by the emotion recognition means, and order means for ordering the ingredients suggested by the suggestion means through a delivery service. This enables efficient menu planning based on the user's emotional state and selection and ordering of necessary ingredients.

[0360] A "terminal device" is an electronic device used to transmit ingredient information entered by the user to a server.

[0361] A "server means" is a central processing unit that receives ingredient information transmitted from terminal means and stores it in a database.

[0362] The "menu generation means" is a function in which the server means generates a weekly menu based on stored ingredient information.

[0363] The "method for listing missing ingredients" is a function that lists any missing ingredients needed for the menu generated by the menu generation method.

[0364] "Display means" refers to a function that displays to the user the list of missing ingredients compiled by the missing ingredient list means, as well as the weekly menu.

[0365] "Emotion recognition means" refers to a function that recognizes the user's emotional state using an emotion engine provided in the server.

[0366] The "suggestion mechanism" is a function that makes suggestions based on the user's emotional state as recognized by the emotion recognition mechanism.

[0367] The "ordering method" refers to the function of ordering the ingredients suggested by the suggestion method through a delivery service.

[0368] System Configuration

[0369] To realize this invention, a system including the following hardware and software is required.

[0370] 1. Terminal device: This is a device for the user to input information about the food items currently in the refrigerator, and is implemented through a smartphone or tablet application.

[0371] 2. Server method: Receive ingredient information sent from the terminal and store it in a database. This can be done using a general cloud server or a local server.

[0372] 3. Database: Manages stored ingredient and recipe information, using SQL or NoSQL databases.

[0373] 4. Menu generation method: This is a server-based function that generates weekly menus based on ingredient information stored in a database and receiving feedback from the emotion engine.

[0374] 5. Ingredient Listing Method: This is a server-based function that compares all the ingredients required for the menu generated by the menu generation method with the ingredient information currently held by the user, and lists any missing ingredients.

[0375] 6. Display means: A terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[0376] 7. Emotion Recognition Means: An emotion engine is used to analyze the user's emotions, and the results are fed back to the menu generation means.

[0377] 8. Proposed method: Based on the user's emotional information obtained by the emotion recognition method, the system provides the user with the most suitable meal suggestions.

[0378] 9. Ordering method: This is a function that allows users to order the suggested ingredients through a delivery service.

[0379] Usage example

[0380] 1. User input and submission of ingredient information:

[0381] The user uses a smartphone app to input information about the ingredients they currently possess, such as "salmon," "chicken," "cabbage," and "carrots." The device then sends this information to the server.

[0382] 2. Preservation of ingredient information:

[0383] The server receives ingredient information sent from the terminal and stores it in the database. This allows for centralized management of the ingredient information held by users.

[0384] 3. Analysis of emotional data:

[0385] While the user is using the app, the emotion engine analyzes the user's facial expressions and voice to recognize their current emotional state. Specifically, it detects emotions such as "tired," "stressed," and "happy."

[0386] 4. Menu generation:

[0387] Based on the stored ingredient information and feedback from the emotion engine, the server selects appropriate recipes from the recipe database and generates a weekly meal plan. For example, if the user is tired, it will suggest a meal to replenish their energy.

[0388] 5. List of missing ingredients:

[0389] All the ingredients needed for the menu are identified, and by comparing them with the user's current ingredient list, any missing ingredients are highlighted. For example, "soy sauce" or "chicken broth."

[0390] 6. Displaying the results:

[0391] The generated weekly meal plan and a list of missing ingredients are displayed to the user via a smartphone app. For example, it might say, "Monday: Grilled salmon and vegetables to replenish energy," or "Tuesday: Relaxing chicken dish."

[0392] 7. Ordering missing ingredients:

[0393] Based on the displayed list of missing ingredients, users can order the necessary ingredients through a delivery service.

[0394] Examples of specific cases and prompt statements

[0395] Specific example:

[0396] The user enters "salmon" and "cabbage" on their smartphone.

[0397] The terminal sends ingredient information to the server.

[0398] The server saves ingredient information to the database.

[0399] The emotion engine recognizes from the camera footage that the user is "tired."

[0400] The app suggests "grilled salmon and stir-fried vegetables" as a suitable meal for nutritional supplementation.

[0401] The list of missing ingredients includes "soy sauce".

[0402] I ordered the soy sauce I was running low on through a delivery service.

[0403] Example of a prompt:

[0404] User-input ingredient information: ["Salmon", "Cabbage"]

[0405] User's emotional state: "Tired"

[0406] Suggested menu: "Grilled salmon and stir-fried vegetables"

[0407] List of missing ingredients: ["Soy sauce"]

[0408] Delivery order for missing ingredients: ["Soy sauce"]

[0409] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0410] Step 1:

[0411] The user uses a smartphone application to input information about the ingredients they currently possess. For example, they might input ingredients such as "salmon," "chicken," "cabbage," and "carrots." The entered ingredient information is then sent from the device to the server. The input is in text format, and this text data is sent to the server.

[0412] Step 2:

[0413] The server stores ingredient information sent from the terminal into an SQL database. The database stores and manages ingredient information along with user-specific identification information. This allows for centralized management of the ingredient information each user possesses.

[0414] Step 3:

[0415] While the user is using the application, the smartphone's camera and microphone are used to perform emotion recognition. The emotion engine analyzes the input audio and video data to recognize the user's emotional state. For example, it may recognize emotions such as "tired," "stressed," or "happy." This emotion information is sent to the server as text data.

[0416] Step 4:

[0417] The server selects an appropriate recipe from the recipe database based on emotional data sent from the emotion engine and ingredient information stored in the database. The menu generation system selects recipes based on different themes for each day of the week and automatically generates a week's worth of menus. The generated menus are saved in text format as a list of dish names and required ingredients.

[0418] Step 5:

[0419] The server lists all the necessary ingredients based on the menu generated by the menu generation system. Next, it compares the listed ingredients with the user's ingredient information stored in the database to identify any missing ingredients. For example, "soy sauce" and "chicken broth" might be listed as missing ingredients. This information is also saved as text data.

[0420] Step 6:

[0421] The server sends the generated weekly meal plan and list of missing ingredients to the user's device, displaying them visually. Users can check the meal plan and list of missing ingredients through a smartphone application. For example, specific meal plans such as "Monday: Grilled salmon and vegetables to replenish energy" and "Tuesday: Relaxing chicken dish" are displayed.

[0422] Step 7:

[0423] Users can order missing ingredients through a delivery service based on the displayed list of missing ingredients. The ordering system automatically confirms the order in conjunction with the delivery service's API and delivers the ingredients at the user's desired time. For example, if "Missing: Soy sauce" and "Missing: Chicken broth" are ordered through the delivery service, the items will be delivered.

[0424] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0425] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0426] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0427] [Second Embodiment]

[0428] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0429] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0430] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0431] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0432] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0433] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0434] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0435] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0436] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0437] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0438] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0439] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0440] overview

[0441] This invention is a system that automatically generates an optimal menu based on the user's ingredient information, lists any missing ingredients, and provides them to the user. This system enables users to efficiently plan meals and simplify grocery shopping.

[0442] System Configuration

[0443] This system includes the following components:

[0444] 1. Terminal: An electronic device used by the user to input ingredient information and send it to the server.

[0445] 2. Server: A central processing unit that receives ingredient information sent from terminals and stores it in a database.

[0446] 3. Database: A system for managing stored information on ingredients and recipes.

[0447] 4. Menu generation method: A server-based function that automatically generates a week's worth of menus based on stored ingredient information.

[0448] 5. Method for listing missing ingredients: A server-side function that compares all the ingredients required for the generated menu with the ingredients the user currently possesses and lists any missing ingredients.

[0449] 6. Display means: A terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[0450] Program processing

[0451] Enter and submit ingredient information

[0452] The user enters information about the food items currently in their refrigerator into the terminal. The terminal converts this information into a data structure and sends it to the server. For example, if the user has "salmon," "chicken," "cabbage," and "carrots," the terminal will send this data.

[0453] Food ingredient information storage

[0454] The server receives ingredient information sent from the terminal and stores it in the database along with the user's identification information. This allows for centralized management of the ingredient information held by each user.

[0455] Menu generation

[0456] The server selects appropriate recipes from its recipe database based on saved ingredient information. It searches for recipes based on different themes for each day of the week (e.g., fish dishes on Mondays, meat dishes on Tuesdays) and generates a week's worth of meal plans.

[0457] List of missing ingredients

[0458] The server identifies all the ingredients needed for the generated weekly menu and compares them to the ingredients the user currently possesses. Based on the comparison, it lists any missing ingredients.

[0459] Displaying Results

[0460] The terminal displays the user a weekly menu and a list of missing ingredients received from the server. This allows users to plan their meals and makes it easier to buy the necessary ingredients.

[0461] Specific example

[0462] User: Uses a smartphone to input "salmon," "chicken," "cabbage," and "carrots."

[0463] Terminal: Sends the entered ingredient information to the server.

[0464] Server: Receives ingredient information and saves it to the database.

[0465] Server: Generates a weekly menu. For example, recipes like "Grilled Salmon and Vegetables" might be selected for Monday, and "Stir-fried Chicken" for Tuesday.

[0466] Server: Lists the missing ingredients. For example, it might find that "soy sauce" and "chicken broth" are missing.

[0467] Terminal: Displays the generated weekly menu and a list of missing ingredients to the user.

[0468] In this way, the system efficiently supports users in their meal planning and shopping.

[0469] The following describes the processing flow.

[0470] Step 1:

[0471] The user enters the ingredients currently in their refrigerator into the terminal. For example, they might enter "salmon," "chicken," "cabbage," and "carrots."

[0472] Step 2:

[0473] The terminal sends the entered ingredient information to the server. At this time, the ingredient information, along with the user ID, is converted into a data structure such as JSON format.

[0474] Step 3:

[0475] The server receives ingredient information sent from the terminal. The received data is stored in a database using the user ID as the key.

[0476] Step 4:

[0477] The server uses the stored ingredient information from the database to search for appropriate recipes in the recipe database and generate a weekly meal plan. Recipes are selected based on different themes for each day of the week (e.g., fish dishes on Monday, meat dishes on Tuesday).

[0478] Step 5:

[0479] The server identifies all the ingredients needed for the generated weekly menu. This list is constructed based on recipe information stored in the recipe database.

[0480] Step 6:

[0481] The server compares the list of required ingredients with the ingredients the user currently possesses. Based on the comparison results, it lists the ingredients that are missing.

[0482] Step 7:

[0483] The terminal displays the weekly menu and a list of missing ingredients received from the server to the user. The display is in a visually easy-to-understand format. For example, it might show menus such as "Monday: Grilled salmon and vegetables" and "Tuesday: Stir-fried chicken," along with missing ingredients such as "Soy sauce, chicken broth."

[0484] (Example 1)

[0485] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0486] In today's busy lifestyle, it's not easy for users to efficiently plan meals and shop for groceries. Planning daily meals and checking for missing ingredients is time-consuming and laborious, making it a struggle for many. Therefore, there is a need for a system that automatically generates optimal meal plans based on the user's existing ingredient information and lists any missing ingredients.

[0487] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0488] In this invention, the server includes an electronic device means for transmitting ingredient information entered by the user to the server, a central processing unit means for receiving the ingredient information transmitted from the electronic device means and storing it in a data structure, a menu generation function that automatically generates a weekly menu based on the stored ingredient information, a comparison calculation means for listing any missing ingredients required for the menu generated by the menu generation function, and a visual display means for displaying the list of missing ingredients and the weekly menu listed by the comparison calculation means to the user. As a result, the user can efficiently plan a weekly menu, easily identify any missing ingredients, and create a shopping list.

[0489] "Ingredient information" refers to information about ingredients that the user possesses, including details such as the name of the ingredient, quantity, and expiration date.

[0490] "Electronic device means" refers to a device used by the user to input ingredient information and transmit that information to a server, and is an electronic device such as a smartphone or tablet.

[0491] The "central processing unit" is a server that has the function of receiving food ingredient information transmitted from the electronic device and storing it in a database.

[0492] "Data structure" refers to the format and structure of the data that the server uses to store the transmitted ingredient information.

[0493] The "menu generation function" is a server-based feature that automatically generates a week's worth of menus based on saved ingredient information.

[0494] The "comparison calculation means" is a server-based processing function that compares all the ingredients required for the generated menu with the ingredients the user currently possesses, and identifies any missing ingredients.

[0495] "Visual display means" refers to a terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[0496] 1. Overview

[0497] This invention is a system that automatically generates an optimal menu based on the user's available ingredients and lists any missing ingredients, providing them to the user. This system allows users to efficiently plan meals and simplify grocery shopping.

[0498] 2. System Configuration

[0499] This system includes the following components:

[0500] 1. Terminal: An electronic device (e.g., smartphone) used by the user to input ingredient information and send it to the server.

[0501] 2. Server: A central processing unit that receives ingredient information sent from terminals and stores it in a data structure.

[0502] 3. Database: A system for managing stored information on ingredients and recipes.

[0503] 4. Menu generation function: A server-based function that automatically generates a week's worth of menus based on saved ingredient information.

[0504] 5. Comparison calculation means: A function that compares all the ingredients required for the generated menu with the ingredients the user currently possesses and lists the missing ingredients.

[0505] 6. Visual display means: A terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[0506] 3. Enter and submit ingredient information

[0507] The user enters information about the food items in their refrigerator at home into a device such as a smartphone. For example, if the user has "salmon," "chicken," "cabbage," and "carrots," they enter the names of these food items using a dedicated application on their device. The device converts the entered food information into JSON data format and sends it to the server using the HTTPS protocol.

[0508] 4. Preservation of ingredient information

[0509] The server receives JSON data sent from the terminal. The received ingredient information is stored in the database along with the user's identification information. If new ingredient information duplicates existing information, the data is kept up-to-date by updating the quantity and expiration date.

[0510] 5. Menu creation

[0511] The server searches for appropriate recipes from the recipe database based on ingredient information stored in the database. It filters recipes based on themes for each day of the week (e.g., fish dishes on Mondays, meat dishes on Tuesdays) and generates a week's worth of meal plans. This generation process uses a recipe selection algorithm based on a generative AI model.

[0512] 6. List of missing ingredients

[0513] The server creates a list of all necessary ingredients based on the generated weekly menu. It compares this list to the user's current ingredient list and lists any missing ingredients, including the quantity and type of each ingredient. For example, it might list ingredients the user doesn't have, such as soy sauce or chicken broth.

[0514] 7. Displaying the results

[0515] The server sends the generated weekly meal plan and a list of missing ingredients to the device in JSON format. The device parses the received JSON data and displays it visually to the user. Specifically, it displays the weekly meal plan and the list of missing ingredients in the application's UI.

[0516] 8. Specific Examples

[0517] User: Use a smartphone app to input "salmon," "chicken," "cabbage," and "carrots."

[0518] Terminal: Converts the entered information into JSON data and sends it to the server.

[0519] Server: Receives data and saves it to the database.

[0520] Server: Based on the saved data, it selects recipes such as "Grilled Salmon and Vegetables" and "Stir-fried Chicken" from the recipe database and generates a week's worth of meal plans.

[0521] Server: Based on the generated menu, it lists any missing ingredients such as "soy sauce" or "chicken broth".

[0522] Terminal: Displays the generated menu and a list of missing ingredients to the user.

[0523] Examples of input prompts for a generative AI model

[0524] The following is an example of a prompt to input into the generating AI model:

[0525] "Please create a one-week menu using salmon, chicken, cabbage, and carrots. Also, please list any ingredients that are missing from the menu."

[0526] In this way, the system efficiently supports users in their meal planning and shopping.

[0527] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0528] Step 1:

[0529] Enter and submit ingredient information

[0530] Users enter information about the ingredients they have in their refrigerator at home into a dedicated application on their smartphone.

[0531] Input: The user enters "salmon", "chicken", "cabbage", and "carrots".

[0532] The terminal converts the entered ingredient information into JSON data format. This data includes details such as the name of the ingredient, quantity, and expiration date.

[0533] Data processing: Converting ingredient information to JSON data format.

[0534] Output: Converted JSON data.

[0535] The terminal sends the converted JSON data to the server using the HTTPS protocol.

[0536] Function: Generate and send JSON data.

[0537] Step 2:

[0538] Food ingredient information storage

[0539] The server receives JSON data sent from the terminal. This is done along with the user's identification information.

[0540] Input: JSON data received via the HTTPS protocol.

[0541] The server parses the received JSON data and stores information about each ingredient as a separate entry in the database.

[0542] Data processing: Parsing JSON data and saving it to a database.

[0543] Output: Results of saving the analyzed food ingredient information to the database.

[0544] Function: Saves ingredient information.

[0545] Step 3:

[0546] Menu generation

[0547] The server retrieves all the ingredient information the user possesses from the database. This means that it retrieves the necessary data based on the stored ingredient information.

[0548] Input: Ingredient information stored in the database.

[0549] The server filters recipes from the recipe database based on themes for each day of the week and generates a week's worth of meal plans.

[0550] Data processing: Recipe search and filtering based on ingredient information.

[0551] Output: A weekly meal plan has been generated.

[0552] Function: Recipe selection and menu generation using a generative AI model.

[0553] Step 4:

[0554] List of missing ingredients

[0555] The server creates a list of all necessary ingredients based on the generated weekly menu.

[0556] Input: A generated weekly meal plan.

[0557] The server compares the user's current ingredient list with the generated ingredient list and lists any missing ingredients.

[0558] Data processing: Compare the list of ingredients you have with the list of ingredients you need.

[0559] Output: A list of missing ingredients.

[0560] Operation: Lists missing ingredients through comparison operations.

[0561] Step 5:

[0562] Displaying Results

[0563] The server sends the generated weekly meal plan and a list of missing ingredients to the terminal in JSON format.

[0564] Input: JSON data containing a weekly meal plan and a list of missing ingredients.

[0565] The device parses the received JSON data and displays it visually to the user through the application's UI.

[0566] Data processing: Parsing JSON data and converting it to a display format.

[0567] Output: Weekly meal plan and list of missing ingredients in a displayable format.

[0568] Users can view menus and lists of missing ingredients through their device screen to plan meals in an organized manner.

[0569] Function: Visual display and user confirmation of analyzed data.

[0570] (Application Example 1)

[0571] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0572] Traditional food information management systems generate menus based on the user's existing food information and list missing ingredients. However, they cannot notify users of missing ingredients in real time while shopping, making efficient shopping difficult. As a result, users may overlook necessary ingredients when shopping in physical stores. Furthermore, many systems lack features to guide users to the location of desired ingredients, contributing to prolonged shopping times. Consequently, planned shopping becomes difficult, leading to the problem of unnecessary purchases.

[0573] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0574] In this invention, the server includes terminal means for transmitting ingredient information entered by the user to the server, server means for receiving ingredient information transmitted from the terminal means and storing it in a database, menu generation means for generating a weekly menu based on the stored ingredient information, ingredient shortage list means for listing any missing ingredients needed for the menu generated by the menu generation means, display means for displaying the ingredient shortage list and the weekly menu listed by the ingredient shortage list means to the user, and real-time shopping support means for notifying the user of any missing ingredients in real time when they visit a store and supporting their purchase. As a result, the user can grasp in real time any missing ingredients while shopping at a physical store, enabling them to shop efficiently without missing any necessary ingredients.

[0575] A "terminal device" is an electronic device used to transmit ingredient information entered by the user to a server.

[0576] The "server means" is a central processing unit that receives ingredient information transmitted from the terminal means and stores it in a database.

[0577] The "menu generation means" is a function that automatically generates a weekly menu based on the stored ingredient information of the server means.

[0578] The "means for listing missing ingredients" is a function that lists any missing ingredients needed for the menu generated by the menu generation means.

[0579] "Display means" refers to a device for visually displaying to the user the list of insufficient ingredients and the weekly menu, which have been listed by the insufficient ingredient listing means.

[0580] A "real-time shopping support system" is a function that notifies users in real time of any missing ingredients when they visit a store, and supports their purchase.

[0581] A "database" is a collection of data used to manage information about ingredients and recipes.

[0582] "Comparing" means comparing the ingredients the user currently has with all the ingredients needed for the weekly meal plan to identify any missing ingredients.

[0583] overview

[0584] This invention is a system that automatically generates an optimal menu based on the user's ingredient information, lists any missing ingredients, and provides them to the user. This system enables users to efficiently plan meals and shop for groceries easily. Furthermore, real-time shopping support in stores allows users to purchase necessary ingredients efficiently without missing any.

[0585] System Configuration

[0586] This system includes the following components:

[0587] 1. Terminal device: An electronic device used by the user to input ingredient information and send it to the server. A smartphone is a specific example.

[0588] 2. Server: A central processing unit that receives ingredient information transmitted from terminals and stores it in a database.

[0589] 3. Database: A system for managing stored ingredient information and recipe information. Database management systems such as SQLite are used.

[0590] 4. Menu Generation Method: This is a server-based function that automatically generates a week's worth of menus based on stored ingredient information. It uses a generation AI model that takes into account user preferences and allergy information.

[0591] 5. Method for listing missing ingredients: This is a server-side function that compares all the ingredients required for the generated menu with the ingredients the user currently possesses and lists any missing ingredients.

[0592] 6. Display means: A terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[0593] 7. Real-time shopping support: This function notifies users in real time of any missing ingredients when they visit a store, and supports their purchase.

[0594] Program processing

[0595] Enter and submit ingredient information

[0596] The user uses their smartphone to input information about the food items currently in their refrigerator. This includes barcode scanning and manual entry. The entered food information is converted into a data structure and sent to the server.

[0597] Food ingredient information storage

[0598] The server receives ingredient information sent from the terminal and stores it in the database along with the user's identification information. This allows for centralized management of the ingredient information held by each user.

[0599] Menu generation

[0600] The server selects appropriate recipes from its recipe database based on stored ingredient information. It searches for recipes based on different themes for each day of the week (e.g., fish dishes on Mondays, meat dishes on Tuesdays) and generates a week's worth of meal plans. A generation AI model is used to take user preferences and allergy information into consideration.

[0601] List of missing ingredients

[0602] The server identifies all the ingredients needed for the generated weekly menu and compares them to the ingredients the user currently possesses. Based on the comparison, it lists any missing ingredients.

[0603] Displaying Results

[0604] The terminal displays the user a weekly menu and a list of missing ingredients received from the server. This allows users to plan their meals and makes it easier to buy the necessary ingredients.

[0605] Real-time shopping support

[0606] When users visit a store, they can use their smartphones to check in real time which ingredients are running low. Furthermore, the system can use in-store location information to guide them to the shelf where their desired ingredients are located.

[0607] Specific example

[0608] User: Uses a smartphone to input "chicken," "grilled salmon," "cabbage," and "carrots."

[0609] Server: Receives ingredient information and saves it to the database.

[0610] Server: Uses a generation AI model to generate a weekly menu, such as "Grilled salmon and vegetables" for Monday and "Stir-fried chicken" for Tuesday.

[0611] Server: Lists the missing ingredients. For example, it might find that "onions" and "soy sauce" are missing.

[0612] Display method: The generated weekly menu and a list of missing ingredients are displayed to the user.

[0613] Real-time shopping support: Displays in real time which ingredients are missing at the store the user is visiting, and guides the user to the shelf location using location information.

[0614] Example of a prompt

[0615] "Please list the ingredients the user currently possesses. Next, list all the ingredients needed for the generated weekly meal plan, and then compare and list any missing ingredients."

[0616] In this way, the system efficiently supports users in their meal planning and shopping.

[0617] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0618] Step 1: Enter and submit ingredient information.

[0619] Users input information about ingredients they have at home using their smartphones. Input methods include barcode scanning and manual entry. Specifically, users can scan the barcodes on ingredients using their smartphone's camera or enter text within the app. The entered information is converted into a data structure and sent to the server in JSON format. <Input: Ingredient information (e.g., chicken, cabbage) / Output: Ingredient data in JSON format>

[0620] Step 2: Save ingredient information

[0621] The server receives ingredient information sent from the terminal and stores it in the database along with the user's identification information. The database used here is SQLite. Specifically, the server parses the received JSON data, converts each ingredient's information into the appropriate format, and inserts it into the database. <Input: Ingredient data in JSON format / Output: Ingredient information stored in the database>

[0622] Step 3: Menu Generation

[0623] The server selects appropriate recipes from a recipe database based on ingredient information stored in the database. It uses a generative AI model to generate a week's worth of menus based on a daily theme (e.g., fish dishes on Mondays, meat dishes on Tuesdays). Specifically, the server selects the most suitable recipes considering the user's preferences and allergy information to create a weekly menu. <Input: Ingredient information from the database / Output: Weekly menu list>

[0624] Step 4: List the missing ingredients

[0625] The server identifies all the ingredients needed for the generated weekly menu and compares them with the ingredients the user currently possesses in the database. Based on this comparison, it lists the missing ingredients. Specifically, it matches the list of ingredients included in the menu with the user's ingredient database and extracts only the missing items. <Input: Weekly menu list, ingredient information from the database / Output: List of missing ingredients>

[0626] Step 5: Displaying the results

[0627] The device visually displays a weekly meal plan and a list of missing ingredients, sent from the server, to the user. Specifically, the smartphone app analyzes the received data, formats it for easy viewing on the user interface, and displays it. <Input: Weekly meal plan, list of missing ingredients / Output: Information displayed on the device screen>

[0628] Step 6: Real-time shopping support

[0629] When a user visits a store, they can use their smartphone to check for missing ingredients in real time. This includes a feature that uses in-store location information to guide them to the location of ingredients. Specifically, it uses GPS and in-store beacons to determine the user's current location and displays the optimal route to the shelves where each ingredient is located. <Input: User location information, missing ingredient list / Output: Real-time ingredient location guidance>

[0630] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0631] overview

[0632] This invention combines a system that automatically generates an optimal menu based on the user's ingredient information and lists any missing ingredients with an emotion engine that recognizes the user's emotions. The emotion engine analyzes the user's emotional state and provides a menu based on that state. This system allows users to efficiently plan meals and enjoy meals that suit their emotions.

[0633] System Configuration

[0634] This system includes the following components:

[0635] 1. Terminal: An electronic device used by the user to input ingredient information and send it to the server.

[0636] 2. Server: A central processing unit that receives ingredient information sent from terminals and stores it in a database.

[0637] 3. Database: A system for managing stored information on ingredients and recipes.

[0638] 4. Menu generation method: A server-based function that automatically generates a week's worth of menus based on saved ingredient information and feedback from an emotion engine.

[0639] 5. Method for listing missing ingredients: A server-side function that compares all the ingredients required for the generated menu with the ingredients the user currently possesses and lists any missing ingredients.

[0640] 6. Display means: A terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[0641] 7. Emotion Engine: A system that analyzes the user's emotions and feeds the results back into the menu generation process.

[0642] Program processing

[0643] Enter and submit ingredient information

[0644] The user enters information about the ingredients currently in their refrigerator into the terminal. For example, they might enter "salmon," "chicken," "cabbage," and "carrots." The terminal converts this information into a data structure and sends it to the server.

[0645] Food ingredient information storage

[0646] The server receives ingredient information sent from the terminal and stores it in a database along with the user's identification information. This allows for centralized management of the ingredient information held by each user.

[0647] User emotion recognition

[0648] While the user is using the device, the emotion engine analyzes the user's input and voice to recognize their current emotional state. For example, it can detect emotions such as "tired," "stressed," or "happy."

[0649] Menu generation

[0650] The server uses ingredient information stored in the database and receives feedback from the emotion engine. It then selects appropriate recipes from the recipe database to generate a weekly meal plan. Recipes are selected based on different themes for each day of the week (e.g., fish dishes on Monday, meat dishes on Tuesday), and dishes that reflect the user's emotional state are suggested.

[0651] List of missing ingredients

[0652] The server identifies all the ingredients needed for the generated weekly menu. This list is constructed based on recipe information stored in the recipe database. The system compares the list of required ingredients with the ingredients the user currently possesses, and lists any missing ingredients.

[0653] Displaying Results

[0654] The device displays the user a weekly menu and a list of missing ingredients received from the server, along with suggestions based on the user's emotional state. For example, it might display menus such as "Monday: Grilled salmon and vegetables to replenish energy" and "Tuesday: Relaxing chicken dish," along with a note saying, "Missing: Soy sauce, chicken broth."

[0655] Specific example

[0656] User: Uses a smartphone to input "salmon," "chicken," "cabbage," and "carrots."

[0657] Terminal: Sends the entered ingredient information to the server.

[0658] Server: Receives ingredient information and saves it to the database.

[0659] Emotion Engine: Recognizes that the user is "tired" from their voice input and feeds that information back to the server.

[0660] Server: Based on feedback from the emotion engine, it selects recipes with a refreshing effect and generates a weekly meal plan. For example, Monday might be a "refreshing salad," Tuesday a "relaxing soup," and so on.

[0661] Server: List the missing ingredients and identify the missing ingredients as "soy sauce" and "chicken broth".

[0662] Terminal: Visually displays the generated weekly menu and a list of missing ingredients to the user.

[0663] This system efficiently plans meals while reflecting the user's emotional state and identifies missing ingredients, thereby enriching the user's life.

[0664] The following describes the processing flow.

[0665] Step 1:

[0666] The user enters the ingredients currently in their refrigerator into the terminal. For example, they might enter "salmon," "chicken," "cabbage," and "carrots."

[0667] Step 2:

[0668] The terminal sends the entered ingredient information to the server. At this time, the ingredient information, along with the user ID, is converted into a data structure such as JSON format.

[0669] Step 3:

[0670] The server receives ingredient information sent from the terminal. The received data is stored in a database using the user ID as the key.

[0671] Step 4:

[0672] While the user is using the device, the emotion engine analyzes the user's input and voice. For example, it can recognize emotions such as "tired," "stressed," or "happy."

[0673] Step 5:

[0674] The emotion engine sends the recognized user's emotional state to the server. For example, the data is sent in the format "User ID: user123, Emotional State: Tired".

[0675] Step 6:

[0676] The server receives feedback from the emotion engine based on stored ingredient information in the database. It then selects appropriate recipes from the recipe database based on different themes for each day of the week, generating a weekly menu. For example, a "refreshing recipe" might be selected for Monday, and a "relaxing recipe" for Tuesday.

[0677] Step 7:

[0678] The server generates a list of all the ingredients needed for the generated week's menu. This is done based on recipe information stored in the recipe database.

[0679] Step 8:

[0680] The server compares the created list of required ingredients with the ingredient information the user possesses. Based on the comparison results, it lists the ingredients that are missing.

[0681] Step 9:

[0682] The terminal displays the user a weekly menu and a list of missing ingredients received from the server. For example, the menu might be displayed in the format of "Monday: Grilled salmon and vegetables to replenish energy" and "Tuesday: Relaxing chicken dish," with "soy sauce" and "chicken broth" being indicated as missing ingredients.

[0683] (Example 2)

[0684] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0685] In today's busy lifestyle, there is a need to efficiently manage ingredients and plan meals, but current systems have difficulty suggesting meals that take into account the user's emotional state. As a result, users may not be able to eat meals that suit their specific emotional state, which could lead to decreased satisfaction with meals. This invention aims to achieve more personalized meal suggestions and necessary ingredient management by taking the user's emotional state into consideration.

[0686] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0687] In this invention, the server includes terminal means for transmitting ingredient information entered by the user to the server, server means for receiving ingredient information transmitted from the terminal means and storing it in a database, menu generation means for generating a weekly menu based on the ingredient information stored by the server means, emotion analysis means connected to the server means for analyzing the user's emotional state and providing feedback thereto to the menu generation means, ingredient deficiency listing means for listing any missing ingredients needed for the menu generated by the menu generation means, and display means for displaying the ingredient deficiency list and the weekly menu listed by the ingredient deficiency listing means to the user. This makes it possible to suggest personalized menus that take into account the user's emotional state and to list any missing ingredients.

[0688] A "terminal device" is an electronic device used to transmit ingredient information entered by the user to a server.

[0689] A "server means" is a central processing unit that receives ingredient information transmitted from terminal means and stores it in a database.

[0690] A "menu generation means" is a means that has the function of generating a weekly menu based on the ingredient information stored by the server means.

[0691] The "emotion analysis means" is a system connected to a server means that analyzes the user's emotional state and provides feedback to the menu generation means.

[0692] The "method for listing missing ingredients" is a method for listing the missing ingredients needed for the menu generated by the menu generation method.

[0693] "Display means" refers to the terminal's function for displaying to the user the list of missing ingredients compiled by the missing ingredient list means, as well as the weekly menu.

[0694] Modes for carrying out the invention

[0695] This invention combines a system that automatically generates an optimal menu based on ingredient information entered by the user and lists any missing ingredients with an emotion analysis means that recognizes the user's emotions. The following describes in detail how this system works.

[0696] System Components

[0697] This system includes the following components:

[0698] 1. Terminal device: An electronic device used by the user to input ingredient information and send it to the server. Examples include smartphones, tablets, and personal computers.

[0699] 2. Server: A central processing unit that receives ingredient information and stores it in a database. The server uses an SQL database to store ingredient information.

[0700] 3. Database: A system for managing stored ingredient information and recipe information. Examples include database management systems such as MySQL and PostgreSQL.

[0701] 4. Menu generation method: A server-based function that automatically generates a week's worth of menus based on stored ingredient information and feedback from an emotion analysis method.

[0702] 5. Emotion Analysis System: A system that analyzes the user's emotional state and provides the results to the menu generation system. Natural language processing technology is used for emotion analysis.

[0703] 6. Method for listing missing ingredients: A server-side function that compares all the ingredients required for the generated menu with the ingredients the user currently possesses and lists any missing ingredients.

[0704] 7. Display method: A terminal function that visually displays the generated weekly menu and list of missing ingredients to the user. This may be done using a smartphone app or a web browser.

[0705] System operation

[0706] 1. The user enters information about the ingredients in their refrigerator into the device. Specifically, they open the smartphone app and enter ingredients such as "salmon," "chicken," "cabbage," and "carrots."

[0707] 2. The terminal converts this input information into a data structure (e.g., JSON format) and sends it to the server. The transmission uses an HTTP POST request.

[0708] 3. The server analyzes the received data and stores it in the database along with the user's identification information. This allows for centralized management of ingredient information for each user.

[0709] 4. When a user inputs an emotional state such as "I'm tired today" via voice input, the terminal analyzes this voice and sends it to an emotion analysis system. The emotion analysis system uses natural language processing technology to analyze this information and provides feedback on the emotional state, such as "tired."

[0710] 5. Based on ingredient information and feedback from emotion analysis, the server selects appropriate recipes from the recipe database and generates a weekly menu. The menu consists of different themes for each day of the week, and dishes are suggested based on the user's emotional state.

[0711] 6. The server lists all the ingredients needed for the week's menu and compares this to the ingredients the user currently possesses. Missing items are listed, and items such as "soy sauce" and "chicken broth" are identified.

[0712] 7. The device visually displays the generated weekly meal plan and a list of missing ingredients. For example, it might say, "Monday: Grilled salmon and vegetables to replenish energy," or "Tuesday: Relaxing chicken dish."

[0713] Examples of specific cases and prompt statements

[0714] Example: A user inputs "salmon," "chicken," "cabbage," and "carrots" from their refrigerator using their smartphone, and enters "tired" as their current emotional state. The system then selects recipes with a refreshing effect and generates a week's worth of meal plans.

[0715] Example of a prompt:

[0716] "I have salmon, chicken, cabbage, and carrots in my refrigerator. I'm also very tired right now. Based on that, please suggest a meal plan for the week and tell me what ingredients I'm missing."

[0717] This system allows users to easily create optimal menus that reflect their emotional state and automatically identify any missing ingredients.

[0718] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0719] Step 1:

[0720] The user enters food information on their device. The user opens the app on their smartphone and enters the food items they have in their refrigerator, such as "salmon," "chicken," "cabbage," and "carrots."

[0721] Input: The name of the ingredient entered by the user.

[0722] Output: The input data is saved to the terminal.

[0723] Specific action: The user enters the name of the ingredient into a text input field in the smartphone app and presses the "Submit" button.

[0724] Step 2:

[0725] The terminal converts the entered ingredient information into a data structure and sends it to the server. The terminal converts the entered text information into JSON format and sends an HTTP POST request to the server.

[0726] Input: The name of the ingredient entered by the user.

[0727] Output: Ingredient information sent to the server (in JSON format).

[0728] Specific operation: The program converts the ingredient information entered on the terminal into JSON format and sends an HTTP POST request to the URL endpoint.

[0729] Step 3:

[0730] The server receives the submitted ingredient information and stores it in the database. The server receives an HTTP request, parses the JSON data, and stores it in the SQL database.

[0731] Input: Ingredient information (in JSON format) sent from the device.

[0732] Output: Ingredient information stored in the database.

[0733] Specific operation: The server receives ingredient information using a REST API, parses it, and executes an INSERT query to insert it into the database along with the user's identification information, and then saves it.

[0734] Step 4:

[0735] The user enters their emotional state. They use their device to voice-input "I'm tired today."

[0736] Input: User's emotional state.

[0737] Output: The input data is saved to the terminal.

[0738] Specific operation: The user uses the voice input function to communicate their emotional state to the device.

[0739] Step 5:

[0740] The device transmits its emotional state to the emotion analysis system. The emotion analysis system uses natural language processing to analyze this information and provides feedback on the emotional state, such as "fatigue."

[0741] Input: User's emotional state.

[0742] Output: Emotional data analyzed by the emotion analysis tool.

[0743] Specific operation: The device converts the audio data into text data, which is then sent to the sentiment analysis engine. The analysis engine uses a natural language processing model to identify the emotional state (e.g., tired) and returns the result.

[0744] Step 6:

[0745] The server generates a menu based on ingredient information and emotional feedback from emotion analysis tools. The server retrieves ingredient information from a database, selects appropriate recipes based on the feedback, and generates a weekly menu.

[0746] Input: Food ingredient information obtained from a database, emotional feedback from emotion analysis tools.

[0747] Output: A week's worth of menus.

[0748] Specific operation: The server retrieves stored ingredient information using SQL queries, selects a suitable recipe from the recipe database considering the sentiment analysis results, and creates a weekly meal plan. For example, for a "tired" state, it selects a recipe that provides energy.

[0749] Step 7:

[0750] The server generates a weekly menu and creates a list of necessary ingredients. It then compares this list with the user's existing ingredient information and lists any missing ingredients.

[0751] Input: Weekly menu, user ingredient information stored in the database.

[0752] Output: List of missing ingredients.

[0753] Specific operation: The server identifies all ingredients included in the week's menu, compares them with the current user's ingredient information using an SQL query, and lists any missing ingredients.

[0754] Step 8:

[0755] The terminal displays the generated weekly menu and a list of missing ingredients to the user. It receives response data from the server and displays it on the user interface.

[0756] Input: Weekly menu and list of missing ingredients from the server.

[0757] Output: Weekly meal plan and list of missing ingredients displayed to the user.

[0758] Specific operation: The device receives a response from the server and displays menus such as "Monday: Grilled salmon and vegetables to replenish energy" and "Tuesday: Relaxing chicken dish" through the application's UI, as well as a list of missing ingredients such as "Missing: Soy sauce, chicken broth".

[0759] (Application Example 2)

[0760] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0761] In modern households, food management and meal planning are not done efficiently, making it particularly difficult to suggest appropriate meals that suit the user's emotional state. Furthermore, the inability to select the optimal ingredients leads to food waste.

[0762] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes terminal means for transmitting ingredient information entered by the user to the server, server means for receiving the ingredient information transmitted from the terminal means and storing it in a database, menu generation means for generating a weekly menu based on the ingredient information stored by the server means, ingredient deficiency listing means for listing any missing ingredients needed for the menu generated by the menu generation means, display means for displaying the ingredient deficiency list and the weekly menu listed by the ingredient deficiency listing means to the user, emotion recognition means for recognizing the user's emotional state using an emotion engine provided in the server means, suggestion means for making suggestions based on the user's emotional state recognized by the emotion recognition means, and order means for ordering the ingredients suggested by the suggestion means through a delivery service. This enables efficient menu planning based on the user's emotional state and selection and ordering of necessary ingredients.

[0763] A "terminal device" is an electronic device used to transmit ingredient information entered by the user to a server.

[0764] A "server means" is a central processing unit that receives ingredient information transmitted from terminal means and stores it in a database.

[0765] The "menu generation means" is a function in which the server means generates a weekly menu based on stored ingredient information.

[0766] The "method for listing missing ingredients" is a function that lists any missing ingredients needed for the menu generated by the menu generation method.

[0767] "Display means" refers to a function that displays to the user the list of missing ingredients compiled by the missing ingredient list means, as well as the weekly menu.

[0768] "Emotion recognition means" refers to a function that recognizes the user's emotional state using an emotion engine provided in the server.

[0769] The "suggestion mechanism" is a function that makes suggestions based on the user's emotional state as recognized by the emotion recognition mechanism.

[0770] The "ordering method" refers to the function of ordering the ingredients suggested by the suggestion method through a delivery service.

[0771] System Configuration

[0772] To realize this invention, a system including the following hardware and software is required.

[0773] 1. Terminal device: This is a device for the user to input information about the food items currently in the refrigerator, and is implemented through a smartphone or tablet application.

[0774] 2. Server method: Receive ingredient information sent from the terminal and store it in a database. This can be done using a general cloud server or a local server.

[0775] 3. Database: Manages stored ingredient and recipe information, using SQL or NoSQL databases.

[0776] 4. Menu generation method: This is a server-based function that generates weekly menus based on ingredient information stored in a database and receiving feedback from the emotion engine.

[0777] 5. Ingredient Listing Method: This is a server-based function that compares all the ingredients required for the menu generated by the menu generation method with the ingredient information currently held by the user, and lists any missing ingredients.

[0778] 6. Display means: A terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[0779] 7. Emotion Recognition Means: An emotion engine is used to analyze the user's emotions, and the results are fed back to the menu generation means.

[0780] 8. Proposed method: Based on the user's emotional information obtained by the emotion recognition method, the system provides the user with the most suitable meal suggestions.

[0781] 9. Ordering method: This is a function that allows users to order the suggested ingredients through a delivery service.

[0782] Usage example

[0783] 1. User input and submission of ingredient information:

[0784] The user uses a smartphone app to input information about the ingredients they currently possess, such as "salmon," "chicken," "cabbage," and "carrots." The device then sends this information to the server.

[0785] 2. Preservation of ingredient information:

[0786] The server receives ingredient information sent from the terminal and stores it in the database. This allows for centralized management of the ingredient information held by users.

[0787] 3. Analysis of emotional data:

[0788] While the user is using the app, the emotion engine analyzes the user's facial expressions and voice to recognize their current emotional state. Specifically, it detects emotions such as "tired," "stressed," and "happy."

[0789] 4. Menu generation:

[0790] Based on the stored ingredient information and feedback from the emotion engine, the server selects appropriate recipes from the recipe database and generates a weekly meal plan. For example, if the user is tired, it will suggest a meal to replenish their energy.

[0791] 5. List of missing ingredients:

[0792] All the ingredients needed for the menu are identified, and by comparing them with the user's current ingredient list, any missing ingredients are highlighted. For example, "soy sauce" or "chicken broth."

[0793] 6. Displaying the results:

[0794] The generated weekly meal plan and a list of missing ingredients are displayed to the user via a smartphone app. For example, it might say, "Monday: Grilled salmon and vegetables to replenish energy," or "Tuesday: Relaxing chicken dish."

[0795] 7. Ordering missing ingredients:

[0796] Based on the displayed list of missing ingredients, users can order the necessary ingredients through a delivery service.

[0797] Examples of specific cases and prompt statements

[0798] Specific example:

[0799] The user enters "salmon" and "cabbage" on their smartphone.

[0800] The terminal sends ingredient information to the server.

[0801] The server saves ingredient information to the database.

[0802] The emotion engine recognizes from the camera footage that the user is "tired."

[0803] The app suggests "grilled salmon and stir-fried vegetables" as a suitable meal for nutritional supplementation.

[0804] The list of missing ingredients includes "soy sauce".

[0805] I ordered the soy sauce I was running low on through a delivery service.

[0806] Example of a prompt:

[0807] User-input ingredient information: ["Salmon", "Cabbage"]

[0808] User's emotional state: "Tired"

[0809] Suggested menu: "Grilled salmon and stir-fried vegetables"

[0810] List of missing ingredients: ["Soy sauce"]

[0811] Delivery order for missing ingredients: ["Soy sauce"]

[0812] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0813] Step 1:

[0814] The user uses a smartphone application to input information about the ingredients they currently possess. For example, they might input ingredients such as "salmon," "chicken," "cabbage," and "carrots." The entered ingredient information is then sent from the device to the server. The input is in text format, and this text data is sent to the server.

[0815] Step 2:

[0816] The server stores ingredient information sent from the terminal into an SQL database. The database stores and manages ingredient information along with user-specific identification information. This allows for centralized management of the ingredient information each user possesses.

[0817] Step 3:

[0818] While the user is using the application, the smartphone's camera and microphone are used to perform emotion recognition. The emotion engine analyzes the input audio and video data to recognize the user's emotional state. For example, it may recognize emotions such as "tired," "stressed," or "happy." This emotion information is sent to the server as text data.

[0819] Step 4:

[0820] The server selects an appropriate recipe from the recipe database based on emotional data sent from the emotion engine and ingredient information stored in the database. The menu generation system selects recipes based on different themes for each day of the week and automatically generates a week's worth of menus. The generated menus are saved in text format as a list of dish names and required ingredients.

[0821] Step 5:

[0822] The server lists all the necessary ingredients based on the menu generated by the menu generation system. Next, it compares the listed ingredients with the user's ingredient information stored in the database to identify any missing ingredients. For example, "soy sauce" and "chicken broth" might be listed as missing ingredients. This information is also saved as text data.

[0823] Step 6:

[0824] The server sends the generated weekly meal plan and list of missing ingredients to the user's device, displaying them visually. Users can check the meal plan and list of missing ingredients through a smartphone application. For example, specific meal plans such as "Monday: Grilled salmon and vegetables to replenish energy" and "Tuesday: Relaxing chicken dish" are displayed.

[0825] Step 7:

[0826] Users can order missing ingredients through a delivery service based on the displayed list of missing ingredients. The ordering system automatically confirms the order in conjunction with the delivery service's API and delivers the ingredients at the user's desired time. For example, if "Missing: Soy sauce" and "Missing: Chicken broth" are ordered through the delivery service, the items will be delivered.

[0827] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0828] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0829] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0830] [Third Embodiment]

[0831] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0832] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0833] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0834] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0835] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0836] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0837] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0838] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0839] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0840] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0841] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0842] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0843] overview

[0844] This invention is a system that automatically generates an optimal menu based on the user's ingredient information, lists any missing ingredients, and provides them to the user. This system enables users to efficiently plan meals and simplify grocery shopping.

[0845] System Configuration

[0846] This system includes the following components:

[0847] 1. Terminal: An electronic device used by the user to input ingredient information and send it to the server.

[0848] 2. Server: A central processing unit that receives ingredient information sent from terminals and stores it in a database.

[0849] 3. Database: A system for managing stored information on ingredients and recipes.

[0850] 4. Menu generation method: A server-based function that automatically generates a week's worth of menus based on stored ingredient information.

[0851] 5. Method for listing missing ingredients: A server-side function that compares all the ingredients required for the generated menu with the ingredients the user currently possesses and lists any missing ingredients.

[0852] 6. Display means: A terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[0853] Program processing

[0854] Enter and submit ingredient information

[0855] The user enters information about the food items currently in their refrigerator into the terminal. The terminal converts this information into a data structure and sends it to the server. For example, if the user has "salmon," "chicken," "cabbage," and "carrots," the terminal will send this data.

[0856] Food ingredient information storage

[0857] The server receives ingredient information sent from the terminal and stores it in the database along with the user's identification information. This allows for centralized management of the ingredient information held by each user.

[0858] Menu generation

[0859] The server selects appropriate recipes from its recipe database based on saved ingredient information. It searches for recipes based on different themes for each day of the week (e.g., fish dishes on Mondays, meat dishes on Tuesdays) and generates a week's worth of meal plans.

[0860] List of missing ingredients

[0861] The server identifies all the ingredients needed for the generated weekly menu and compares them to the ingredients the user currently possesses. Based on the comparison, it lists any missing ingredients.

[0862] Displaying Results

[0863] The terminal displays the user a weekly menu and a list of missing ingredients received from the server. This allows users to plan their meals and makes it easier to buy the necessary ingredients.

[0864] Specific example

[0865] User: Uses a smartphone to input "salmon," "chicken," "cabbage," and "carrots."

[0866] Terminal: Sends the entered ingredient information to the server.

[0867] Server: Receives ingredient information and saves it to the database.

[0868] Server: Generates a weekly menu. For example, recipes like "Grilled Salmon and Vegetables" might be selected for Monday, and "Stir-fried Chicken" for Tuesday.

[0869] Server: Lists the missing ingredients. For example, it might find that "soy sauce" and "chicken broth" are missing.

[0870] Terminal: Displays the generated weekly menu and a list of missing ingredients to the user.

[0871] In this way, the system efficiently supports users in their meal planning and shopping.

[0872] The following describes the processing flow.

[0873] Step 1:

[0874] The user enters the ingredients currently in their refrigerator into the terminal. For example, they might enter "salmon," "chicken," "cabbage," and "carrots."

[0875] Step 2:

[0876] The terminal sends the entered ingredient information to the server. At this time, the ingredient information, along with the user ID, is converted into a data structure such as JSON format.

[0877] Step 3:

[0878] The server receives ingredient information sent from the terminal. The received data is stored in a database using the user ID as the key.

[0879] Step 4:

[0880] The server uses the stored ingredient information from the database to search for appropriate recipes in the recipe database and generate a weekly meal plan. Recipes are selected based on different themes for each day of the week (e.g., fish dishes on Monday, meat dishes on Tuesday).

[0881] Step 5:

[0882] The server identifies all the ingredients needed for the generated weekly menu. This list is constructed based on recipe information stored in the recipe database.

[0883] Step 6:

[0884] The server compares the list of required ingredients with the ingredients the user currently possesses. Based on the comparison results, it lists the ingredients that are missing.

[0885] Step 7:

[0886] The terminal displays the weekly menu and a list of missing ingredients received from the server to the user. The display is in a visually easy-to-understand format. For example, it might show menus such as "Monday: Grilled salmon and vegetables" and "Tuesday: Stir-fried chicken," along with missing ingredients such as "Soy sauce, chicken broth."

[0887] (Example 1)

[0888] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0889] In today's busy lifestyle, it's not easy for users to efficiently plan meals and shop for groceries. Planning daily meals and checking for missing ingredients is time-consuming and laborious, making it a struggle for many. Therefore, there is a need for a system that automatically generates optimal meal plans based on the user's existing ingredient information and lists any missing ingredients.

[0890] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0891] In this invention, the server includes an electronic device means for transmitting ingredient information entered by the user to the server, a central processing unit means for receiving the ingredient information transmitted from the electronic device means and storing it in a data structure, a menu generation function that automatically generates a weekly menu based on the stored ingredient information, a comparison calculation means for listing any missing ingredients required for the menu generated by the menu generation function, and a visual display means for displaying the list of missing ingredients and the weekly menu listed by the comparison calculation means to the user. As a result, the user can efficiently plan a weekly menu, easily identify any missing ingredients, and create a shopping list.

[0892] "Ingredient information" refers to information about ingredients that the user possesses, including details such as the name of the ingredient, quantity, and expiration date.

[0893] "Electronic device means" refers to a device used by the user to input ingredient information and transmit that information to a server, and is an electronic device such as a smartphone or tablet.

[0894] The "central processing unit" is a server that has the function of receiving food ingredient information transmitted from the electronic device and storing it in a database.

[0895] "Data structure" refers to the format and structure of the data that the server uses to store the transmitted ingredient information.

[0896] The "menu generation function" is a server-based feature that automatically generates a week's worth of menus based on saved ingredient information.

[0897] The "comparison calculation means" is a server-based processing function that compares all the ingredients required for the generated menu with the ingredients the user currently possesses, and identifies any missing ingredients.

[0898] "Visual display means" refers to a terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[0899] 1. Overview

[0900] This invention is a system that automatically generates an optimal menu based on the user's available ingredients and lists any missing ingredients, providing them to the user. This system allows users to efficiently plan meals and simplify grocery shopping.

[0901] 2. System Configuration

[0902] This system includes the following components:

[0903] 1. Terminal: An electronic device (e.g., smartphone) used by the user to input ingredient information and send it to the server.

[0904] 2. Server: A central processing unit that receives ingredient information sent from terminals and stores it in a data structure.

[0905] 3. Database: A system for managing stored information on ingredients and recipes.

[0906] 4. Menu generation function: A server-based function that automatically generates a week's worth of menus based on saved ingredient information.

[0907] 5. Comparison calculation means: A function that compares all the ingredients required for the generated menu with the ingredients the user currently possesses and lists the missing ingredients.

[0908] 6. Visual display means: A terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[0909] 3. Enter and submit ingredient information

[0910] The user enters information about the food items in their refrigerator at home into a device such as a smartphone. For example, if the user has "salmon," "chicken," "cabbage," and "carrots," they enter the names of these food items using a dedicated application on their device. The device converts the entered food information into JSON data format and sends it to the server using the HTTPS protocol.

[0911] 4. Preservation of ingredient information

[0912] The server receives JSON data sent from the terminal. The received ingredient information is stored in the database along with the user's identification information. If new ingredient information duplicates existing information, the data is kept up-to-date by updating the quantity and expiration date.

[0913] 5. Menu creation

[0914] The server searches for appropriate recipes from the recipe database based on ingredient information stored in the database. It filters recipes based on themes for each day of the week (e.g., fish dishes on Mondays, meat dishes on Tuesdays) and generates a week's worth of meal plans. This generation process uses a recipe selection algorithm based on a generative AI model.

[0915] 6. List of missing ingredients

[0916] The server creates a list of all necessary ingredients based on the generated weekly menu. It compares this list to the user's current ingredient list and lists any missing ingredients, including the quantity and type of each ingredient. For example, it might list ingredients the user doesn't have, such as soy sauce or chicken broth.

[0917] 7. Displaying the results

[0918] The server sends the generated weekly meal plan and a list of missing ingredients to the device in JSON format. The device parses the received JSON data and displays it visually to the user. Specifically, it displays the weekly meal plan and the list of missing ingredients in the application's UI.

[0919] 8. Specific Examples

[0920] User: Use a smartphone app to input "salmon," "chicken," "cabbage," and "carrots."

[0921] Terminal: Converts the entered information into JSON data and sends it to the server.

[0922] Server: Receives data and saves it to the database.

[0923] Server: Based on the saved data, it selects recipes such as "Grilled Salmon and Vegetables" and "Stir-fried Chicken" from the recipe database and generates a week's worth of meal plans.

[0924] Server: Based on the generated menu, it lists any missing ingredients such as "soy sauce" or "chicken broth".

[0925] Terminal: Displays the generated menu and a list of missing ingredients to the user.

[0926] Examples of input prompts for a generative AI model

[0927] The following is an example of a prompt to input into the generating AI model:

[0928] "Please create a one-week menu using salmon, chicken, cabbage, and carrots. Also, please list any ingredients that are missing from the menu."

[0929] In this way, the system efficiently supports users in their meal planning and shopping.

[0930] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0931] Step 1:

[0932] Enter and submit ingredient information

[0933] Users enter information about the ingredients they have in their refrigerator at home into a dedicated application on their smartphone.

[0934] Input: The user enters "salmon", "chicken", "cabbage", and "carrots".

[0935] The terminal converts the entered ingredient information into JSON data format. This data includes details such as the name of the ingredient, quantity, and expiration date.

[0936] Data processing: Converting ingredient information to JSON data format.

[0937] Output: Converted JSON data.

[0938] The terminal sends the converted JSON data to the server using the HTTPS protocol.

[0939] Function: Generate and send JSON data.

[0940] Step 2:

[0941] Food ingredient information storage

[0942] The server receives JSON data sent from the terminal. This is done along with the user's identification information.

[0943] Input: JSON data received via the HTTPS protocol.

[0944] The server parses the received JSON data and stores information about each ingredient as a separate entry in the database.

[0945] Data processing: Parsing JSON data and saving it to a database.

[0946] Output: Results of saving the analyzed food ingredient information to the database.

[0947] Function: Saves ingredient information.

[0948] Step 3:

[0949] Menu generation

[0950] The server retrieves all the ingredient information the user possesses from the database. This means that it retrieves the necessary data based on the stored ingredient information.

[0951] Input: Ingredient information stored in the database.

[0952] The server filters recipes from the recipe database based on themes for each day of the week and generates a week's worth of meal plans.

[0953] Data processing: Recipe search and filtering based on ingredient information.

[0954] Output: A weekly meal plan has been generated.

[0955] Function: Recipe selection and menu generation using a generative AI model.

[0956] Step 4:

[0957] List of missing ingredients

[0958] The server creates a list of all necessary ingredients based on the generated weekly menu.

[0959] Input: A generated weekly meal plan.

[0960] The server compares the user's current ingredient list with the generated ingredient list and lists any missing ingredients.

[0961] Data processing: Compare the list of ingredients you have with the list of ingredients you need.

[0962] Output: A list of missing ingredients.

[0963] Operation: Lists missing ingredients through comparison operations.

[0964] Step 5:

[0965] Displaying Results

[0966] The server sends the generated weekly meal plan and a list of missing ingredients to the terminal in JSON format.

[0967] Input: JSON data containing a weekly meal plan and a list of missing ingredients.

[0968] The device parses the received JSON data and displays it visually to the user through the application's UI.

[0969] Data processing: Parsing JSON data and converting it to a display format.

[0970] Output: Weekly meal plan and list of missing ingredients in a displayable format.

[0971] Users can view menus and lists of missing ingredients through their device screen to plan meals in an organized manner.

[0972] Function: Visual display and user confirmation of analyzed data.

[0973] (Application Example 1)

[0974] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0975] Traditional food information management systems generate menus based on the user's existing food information and list missing ingredients. However, they cannot notify users of missing ingredients in real time while shopping, making efficient shopping difficult. As a result, users may overlook necessary ingredients when shopping in physical stores. Furthermore, many systems lack features to guide users to the location of desired ingredients, contributing to prolonged shopping times. Consequently, planned shopping becomes difficult, leading to the problem of unnecessary purchases.

[0976] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0977] In this invention, the server includes terminal means for transmitting ingredient information entered by the user to the server, server means for receiving ingredient information transmitted from the terminal means and storing it in a database, menu generation means for generating a weekly menu based on the stored ingredient information, ingredient shortage list means for listing any missing ingredients needed for the menu generated by the menu generation means, display means for displaying the ingredient shortage list and the weekly menu listed by the ingredient shortage list means to the user, and real-time shopping support means for notifying the user of any missing ingredients in real time when they visit a store and supporting their purchase. As a result, the user can grasp in real time any missing ingredients while shopping at a physical store, enabling them to shop efficiently without missing any necessary ingredients.

[0978] A "terminal device" is an electronic device used to transmit ingredient information entered by the user to a server.

[0979] The "server means" is a central processing unit that receives ingredient information transmitted from the terminal means and stores it in a database.

[0980] The "menu generation means" is a function that automatically generates a weekly menu based on the stored ingredient information of the server means.

[0981] The "means for listing missing ingredients" is a function that lists any missing ingredients needed for the menu generated by the menu generation means.

[0982] "Display means" refers to a device for visually displaying to the user the list of insufficient ingredients and the weekly menu, which have been listed by the insufficient ingredient listing means.

[0983] A "real-time shopping support system" is a function that notifies users in real time of any missing ingredients when they visit a store, and supports their purchase.

[0984] A "database" is a collection of data used to manage information about ingredients and recipes.

[0985] "Comparing" means comparing the ingredients the user currently has with all the ingredients needed for the weekly meal plan to identify any missing ingredients.

[0986] overview

[0987] This invention is a system that automatically generates an optimal menu based on the user's ingredient information, lists any missing ingredients, and provides them to the user. This system enables users to efficiently plan meals and shop for groceries easily. Furthermore, real-time shopping support in stores allows users to purchase necessary ingredients efficiently without missing any.

[0988] System Configuration

[0989] This system includes the following components:

[0990] 1. Terminal device: An electronic device used by the user to input ingredient information and send it to the server. A smartphone is a specific example.

[0991] 2. Server: A central processing unit that receives ingredient information transmitted from terminals and stores it in a database.

[0992] 3. Database: A system for managing stored ingredient information and recipe information. Database management systems such as SQLite are used.

[0993] 4. Menu Generation Method: This is a server-based function that automatically generates a week's worth of menus based on stored ingredient information. It uses a generation AI model that takes into account user preferences and allergy information.

[0994] 5. Method for listing missing ingredients: This is a server-side function that compares all the ingredients required for the generated menu with the ingredients the user currently possesses and lists any missing ingredients.

[0995] 6. Display means: A terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[0996] 7. Real-time shopping support: This function notifies users in real time of any missing ingredients when they visit a store, and supports their purchase.

[0997] Program processing

[0998] Enter and submit ingredient information

[0999] The user uses their smartphone to input information about the food items currently in their refrigerator. This includes barcode scanning and manual entry. The entered food information is converted into a data structure and sent to the server.

[1000] Food ingredient information storage

[1001] The server receives ingredient information sent from the terminal and stores it in the database along with the user's identification information. This allows for centralized management of the ingredient information held by each user.

[1002] Menu generation

[1003] The server selects appropriate recipes from its recipe database based on stored ingredient information. It searches for recipes based on different themes for each day of the week (e.g., fish dishes on Mondays, meat dishes on Tuesdays) and generates a week's worth of meal plans. A generation AI model is used to take user preferences and allergy information into consideration.

[1004] List of missing ingredients

[1005] The server identifies all the ingredients needed for the generated weekly menu and compares them to the ingredients the user currently possesses. Based on the comparison, it lists any missing ingredients.

[1006] Displaying Results

[1007] The terminal displays the user a weekly menu and a list of missing ingredients received from the server. This allows users to plan their meals and makes it easier to buy the necessary ingredients.

[1008] Real-time shopping support

[1009] When users visit a store, they can use their smartphones to check in real time which ingredients are running low. Furthermore, the system can use in-store location information to guide them to the shelf where their desired ingredients are located.

[1010] Specific example

[1011] User: Uses a smartphone to input "chicken," "grilled salmon," "cabbage," and "carrots."

[1012] Server: Receives ingredient information and saves it to the database.

[1013] Server: Uses a generation AI model to generate a weekly menu, such as "Grilled salmon and vegetables" for Monday and "Stir-fried chicken" for Tuesday.

[1014] Server: Lists the missing ingredients. For example, it might find that "onions" and "soy sauce" are missing.

[1015] Display method: The generated weekly menu and a list of missing ingredients are displayed to the user.

[1016] Real-time shopping support: Displays in real time which ingredients are missing at the store the user is visiting, and guides the user to the shelf location using location information.

[1017] Example of a prompt

[1018] "Please list the ingredients the user currently possesses. Next, list all the ingredients needed for the generated weekly meal plan, and then compare and list any missing ingredients."

[1019] In this way, the system efficiently supports users in their meal planning and shopping.

[1020] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1021] Step 1: Enter and submit ingredient information.

[1022] Users input information about ingredients they have at home using their smartphones. Input methods include barcode scanning and manual entry. Specifically, users can scan the barcodes on ingredients using their smartphone's camera or enter text within the app. The entered information is converted into a data structure and sent to the server in JSON format. <Input: Ingredient information (e.g., chicken, cabbage) / Output: Ingredient data in JSON format>

[1023] Step 2: Save ingredient information

[1024] The server receives ingredient information sent from the terminal and stores it in the database along with the user's identification information. The database used here is SQLite. Specifically, the server parses the received JSON data, converts each ingredient's information into the appropriate format, and inserts it into the database. <Input: Ingredient data in JSON format / Output: Ingredient information stored in the database>

[1025] Step 3: Menu Generation

[1026] The server selects appropriate recipes from a recipe database based on ingredient information stored in the database. It uses a generative AI model to generate a week's worth of menus based on a daily theme (e.g., fish dishes on Mondays, meat dishes on Tuesdays). Specifically, the server selects the most suitable recipes considering the user's preferences and allergy information to create a weekly menu. <Input: Ingredient information from the database / Output: Weekly menu list>

[1027] Step 4: List the missing ingredients

[1028] The server identifies all the ingredients needed for the generated weekly menu and compares them with the ingredients the user currently possesses in the database. Based on this comparison, it lists the missing ingredients. Specifically, it matches the list of ingredients included in the menu with the user's ingredient database and extracts only the missing items. <Input: Weekly menu list, ingredient information from the database / Output: List of missing ingredients>

[1029] Step 5: Displaying the results

[1030] The device visually displays a weekly meal plan and a list of missing ingredients, sent from the server, to the user. Specifically, the smartphone app analyzes the received data, formats it for easy viewing on the user interface, and displays it. <Input: Weekly meal plan, list of missing ingredients / Output: Information displayed on the device screen>

[1031] Step 6: Real-time shopping support

[1032] When a user visits a store, they can use their smartphone to check for missing ingredients in real time. This includes a feature that uses in-store location information to guide them to the location of ingredients. Specifically, it uses GPS and in-store beacons to determine the user's current location and displays the optimal route to the shelves where each ingredient is located. <Input: User location information, missing ingredient list / Output: Real-time ingredient location guidance>

[1033] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1034] overview

[1035] This invention combines a system that automatically generates an optimal menu based on the user's ingredient information and lists any missing ingredients with an emotion engine that recognizes the user's emotions. The emotion engine analyzes the user's emotional state and provides a menu based on that state. This system allows users to efficiently plan meals and enjoy meals that suit their emotions.

[1036] System Configuration

[1037] This system includes the following components:

[1038] 1. Terminal: An electronic device used by the user to input ingredient information and send it to the server.

[1039] 2. Server: A central processing unit that receives ingredient information sent from terminals and stores it in a database.

[1040] 3. Database: A system for managing stored information on ingredients and recipes.

[1041] 4. Menu generation method: A server-based function that automatically generates a week's worth of menus based on saved ingredient information and feedback from an emotion engine.

[1042] 5. Method for listing missing ingredients: A server-side function that compares all the ingredients required for the generated menu with the ingredients the user currently possesses and lists any missing ingredients.

[1043] 6. Display means: A terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[1044] 7. Emotion Engine: A system that analyzes the user's emotions and feeds the results back into the menu generation process.

[1045] Program processing

[1046] Enter and submit ingredient information

[1047] The user enters information about the ingredients currently in their refrigerator into the terminal. For example, they might enter "salmon," "chicken," "cabbage," and "carrots." The terminal converts this information into a data structure and sends it to the server.

[1048] Food ingredient information storage

[1049] The server receives ingredient information sent from the terminal and stores it in a database along with the user's identification information. This allows for centralized management of the ingredient information held by each user.

[1050] User emotion recognition

[1051] While the user is using the device, the emotion engine analyzes the user's input and voice to recognize their current emotional state. For example, it can detect emotions such as "tired," "stressed," or "happy."

[1052] Menu generation

[1053] The server uses ingredient information stored in the database and receives feedback from the emotion engine. It then selects appropriate recipes from the recipe database to generate a weekly meal plan. Recipes are selected based on different themes for each day of the week (e.g., fish dishes on Monday, meat dishes on Tuesday), and dishes that reflect the user's emotional state are suggested.

[1054] List of missing ingredients

[1055] The server identifies all the ingredients needed for the generated weekly menu. This list is constructed based on recipe information stored in the recipe database. The system compares the list of required ingredients with the ingredients the user currently possesses, and lists any missing ingredients.

[1056] Displaying Results

[1057] The device displays the user a weekly menu and a list of missing ingredients received from the server, along with suggestions based on the user's emotional state. For example, it might display menus such as "Monday: Grilled salmon and vegetables to replenish energy" and "Tuesday: Relaxing chicken dish," along with a note saying, "Missing: Soy sauce, chicken broth."

[1058] Specific example

[1059] User: Uses a smartphone to input "salmon," "chicken," "cabbage," and "carrots."

[1060] Terminal: Sends the entered ingredient information to the server.

[1061] Server: Receives ingredient information and saves it to the database.

[1062] Emotion Engine: Recognizes that the user is "tired" from their voice input and feeds that information back to the server.

[1063] Server: Based on feedback from the emotion engine, it selects recipes with a refreshing effect and generates a weekly meal plan. For example, Monday might be a "refreshing salad," Tuesday a "relaxing soup," and so on.

[1064] Server: List the missing ingredients and identify the missing ingredients as "soy sauce" and "chicken broth".

[1065] Terminal: Visually displays the generated weekly menu and a list of missing ingredients to the user.

[1066] This system efficiently plans meals while reflecting the user's emotional state and identifies missing ingredients, thereby enriching the user's life.

[1067] The following describes the processing flow.

[1068] Step 1:

[1069] The user enters the ingredients currently in their refrigerator into the terminal. For example, they might enter "salmon," "chicken," "cabbage," and "carrots."

[1070] Step 2:

[1071] The terminal sends the entered ingredient information to the server. At this time, the ingredient information, along with the user ID, is converted into a data structure such as JSON format.

[1072] Step 3:

[1073] The server receives ingredient information sent from the terminal. The received data is stored in a database using the user ID as the key.

[1074] Step 4:

[1075] While the user is using the device, the emotion engine analyzes the user's input and voice. For example, it can recognize emotions such as "tired," "stressed," or "happy."

[1076] Step 5:

[1077] The emotion engine sends the recognized user's emotional state to the server. For example, the data is sent in the format "User ID: user123, Emotional State: Tired".

[1078] Step 6:

[1079] The server receives feedback from the emotion engine based on stored ingredient information in the database. It then selects appropriate recipes from the recipe database based on different themes for each day of the week, generating a weekly menu. For example, a "refreshing recipe" might be selected for Monday, and a "relaxing recipe" for Tuesday.

[1080] Step 7:

[1081] The server generates a list of all the ingredients needed for the generated week's menu. This is done based on recipe information stored in the recipe database.

[1082] Step 8:

[1083] The server compares the created list of required ingredients with the ingredient information the user possesses. Based on the comparison results, it lists the ingredients that are missing.

[1084] Step 9:

[1085] The terminal displays the user a weekly menu and a list of missing ingredients received from the server. For example, the menu might be displayed in the format of "Monday: Grilled salmon and vegetables to replenish energy" and "Tuesday: Relaxing chicken dish," with "soy sauce" and "chicken broth" being indicated as missing ingredients.

[1086] (Example 2)

[1087] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1088] In today's busy lifestyle, there is a need to efficiently manage ingredients and plan meals, but current systems have difficulty suggesting meals that take into account the user's emotional state. As a result, users may not be able to eat meals that suit their specific emotional state, which could lead to decreased satisfaction with meals. This invention aims to achieve more personalized meal suggestions and necessary ingredient management by taking the user's emotional state into consideration.

[1089] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1090] In this invention, the server includes terminal means for transmitting ingredient information entered by the user to the server, server means for receiving ingredient information transmitted from the terminal means and storing it in a database, menu generation means for generating a weekly menu based on the ingredient information stored by the server means, emotion analysis means connected to the server means for analyzing the user's emotional state and providing feedback thereto to the menu generation means, ingredient deficiency listing means for listing any missing ingredients needed for the menu generated by the menu generation means, and display means for displaying the ingredient deficiency list and the weekly menu listed by the ingredient deficiency listing means to the user. This makes it possible to suggest personalized menus that take into account the user's emotional state and to list any missing ingredients.

[1091] A "terminal device" is an electronic device used to transmit ingredient information entered by the user to a server.

[1092] A "server means" is a central processing unit that receives ingredient information transmitted from terminal means and stores it in a database.

[1093] A "menu generation means" is a means that has the function of generating a weekly menu based on the ingredient information stored by the server means.

[1094] The "emotion analysis means" is a system connected to a server means that analyzes the user's emotional state and provides feedback to the menu generation means.

[1095] The "method for listing missing ingredients" is a method for listing the missing ingredients needed for the menu generated by the menu generation method.

[1096] "Display means" refers to the terminal's function for displaying to the user the list of missing ingredients compiled by the missing ingredient list means, as well as the weekly menu.

[1097] Modes for carrying out the invention

[1098] This invention combines a system that automatically generates an optimal menu based on ingredient information entered by the user and lists any missing ingredients with an emotion analysis means that recognizes the user's emotions. The following describes in detail how this system works.

[1099] System Components

[1100] This system includes the following components:

[1101] 1. Terminal device: An electronic device used by the user to input ingredient information and send it to the server. Examples include smartphones, tablets, and personal computers.

[1102] 2. Server: A central processing unit that receives ingredient information and stores it in a database. The server uses an SQL database to store ingredient information.

[1103] 3. Database: A system for managing stored ingredient information and recipe information. Examples include database management systems such as MySQL and PostgreSQL.

[1104] 4. Menu generation method: A server-based function that automatically generates a week's worth of menus based on stored ingredient information and feedback from an emotion analysis method.

[1105] 5. Emotion Analysis System: A system that analyzes the user's emotional state and provides the results to the menu generation system. Natural language processing technology is used for emotion analysis.

[1106] 6. Method for listing missing ingredients: A server-side function that compares all the ingredients required for the generated menu with the ingredients the user currently possesses and lists any missing ingredients.

[1107] 7. Display method: A terminal function that visually displays the generated weekly menu and list of missing ingredients to the user. This may be done using a smartphone app or a web browser.

[1108] System operation

[1109] 1. The user enters information about the ingredients in their refrigerator into the device. Specifically, they open the smartphone app and enter ingredients such as "salmon," "chicken," "cabbage," and "carrots."

[1110] 2. The terminal converts this input information into a data structure (e.g., JSON format) and sends it to the server. The transmission uses an HTTP POST request.

[1111] 3. The server analyzes the received data and stores it in the database along with the user's identification information. This allows for centralized management of ingredient information for each user.

[1112] 4. When a user inputs an emotional state such as "I'm tired today" via voice input, the terminal analyzes this voice and sends it to an emotion analysis system. The emotion analysis system uses natural language processing technology to analyze this information and provides feedback on the emotional state, such as "tired."

[1113] 5. Based on ingredient information and feedback from emotion analysis, the server selects appropriate recipes from the recipe database and generates a weekly menu. The menu consists of different themes for each day of the week, and dishes are suggested based on the user's emotional state.

[1114] 6. The server lists all the ingredients needed for the week's menu and compares this to the ingredients the user currently possesses. Missing items are listed, and items such as "soy sauce" and "chicken broth" are identified.

[1115] 7. The device visually displays the generated weekly meal plan and a list of missing ingredients. For example, it might say, "Monday: Grilled salmon and vegetables to replenish energy," or "Tuesday: Relaxing chicken dish."

[1116] Examples of specific cases and prompt statements

[1117] Example: A user inputs "salmon," "chicken," "cabbage," and "carrots" from their refrigerator using their smartphone, and enters "tired" as their current emotional state. The system then selects recipes with a refreshing effect and generates a week's worth of meal plans.

[1118] Example of a prompt:

[1119] "I have salmon, chicken, cabbage, and carrots in my refrigerator. I'm also very tired right now. Based on that, please suggest a meal plan for the week and tell me what ingredients I'm missing."

[1120] This system allows users to easily create optimal menus that reflect their emotional state and automatically identify any missing ingredients.

[1121] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1122] Step 1:

[1123] The user enters food information on their device. The user opens the app on their smartphone and enters the food items they have in their refrigerator, such as "salmon," "chicken," "cabbage," and "carrots."

[1124] Input: The name of the ingredient entered by the user.

[1125] Output: The input data is saved to the terminal.

[1126] Specific action: The user enters the name of the ingredient into a text input field in the smartphone app and presses the "Submit" button.

[1127] Step 2:

[1128] The terminal converts the entered ingredient information into a data structure and sends it to the server. The terminal converts the entered text information into JSON format and sends an HTTP POST request to the server.

[1129] Input: The name of the ingredient entered by the user.

[1130] Output: Ingredient information sent to the server (in JSON format).

[1131] Specific operation: The program converts the ingredient information entered on the terminal into JSON format and sends an HTTP POST request to the URL endpoint.

[1132] Step 3:

[1133] The server receives the submitted ingredient information and stores it in the database. The server receives an HTTP request, parses the JSON data, and stores it in the SQL database.

[1134] Input: Ingredient information (in JSON format) sent from the device.

[1135] Output: Ingredient information stored in the database.

[1136] Specific operation: The server receives ingredient information using a REST API, parses it, and executes an INSERT query to insert it into the database along with the user's identification information, and then saves it.

[1137] Step 4:

[1138] The user enters their emotional state. They use their device to voice-input "I'm tired today."

[1139] Input: User's emotional state.

[1140] Output: The input data is saved to the terminal.

[1141] Specific operation: The user uses the voice input function to communicate their emotional state to the device.

[1142] Step 5:

[1143] The device transmits its emotional state to the emotion analysis system. The emotion analysis system uses natural language processing to analyze this information and provides feedback on the emotional state, such as "fatigue."

[1144] Input: User's emotional state.

[1145] Output: Emotional data analyzed by the emotion analysis tool.

[1146] Specific operation: The device converts the audio data into text data, which is then sent to the sentiment analysis engine. The analysis engine uses a natural language processing model to identify the emotional state (e.g., tired) and returns the result.

[1147] Step 6:

[1148] The server generates a menu based on ingredient information and emotional feedback from emotion analysis tools. The server retrieves ingredient information from a database, selects appropriate recipes based on the feedback, and generates a weekly menu.

[1149] Input: Food ingredient information obtained from a database, emotional feedback from emotion analysis tools.

[1150] Output: A week's worth of menus.

[1151] Specific operation: The server retrieves stored ingredient information using SQL queries, selects a suitable recipe from the recipe database considering the sentiment analysis results, and creates a weekly meal plan. For example, for a "tired" state, it selects a recipe that provides energy.

[1152] Step 7:

[1153] The server generates a weekly menu and creates a list of necessary ingredients. It then compares this list with the user's existing ingredient information and lists any missing ingredients.

[1154] Input: Weekly menu, user ingredient information stored in the database.

[1155] Output: List of missing ingredients.

[1156] Specific operation: The server identifies all ingredients included in the week's menu, compares them with the current user's ingredient information using an SQL query, and lists any missing ingredients.

[1157] Step 8:

[1158] The terminal displays the generated weekly menu and a list of missing ingredients to the user. It receives response data from the server and displays it on the user interface.

[1159] Input: Weekly menu and list of missing ingredients from the server.

[1160] Output: Weekly meal plan and list of missing ingredients displayed to the user.

[1161] Specific operation: The device receives a response from the server and displays menus such as "Monday: Grilled salmon and vegetables to replenish energy" and "Tuesday: Relaxing chicken dish" through the application's UI, as well as a list of missing ingredients such as "Missing: Soy sauce, chicken broth".

[1162] (Application Example 2)

[1163] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1164] In modern households, food management and meal planning are not done efficiently, making it particularly difficult to suggest appropriate meals that suit the user's emotional state. Furthermore, the inability to select the optimal ingredients leads to food waste.

[1165] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes terminal means for transmitting ingredient information entered by the user to the server, server means for receiving the ingredient information transmitted from the terminal means and storing it in a database, menu generation means for generating a weekly menu based on the ingredient information stored by the server means, ingredient deficiency listing means for listing any missing ingredients needed for the menu generated by the menu generation means, display means for displaying the ingredient deficiency list and the weekly menu listed by the ingredient deficiency listing means to the user, emotion recognition means for recognizing the user's emotional state using an emotion engine provided in the server means, suggestion means for making suggestions based on the user's emotional state recognized by the emotion recognition means, and order means for ordering the ingredients suggested by the suggestion means through a delivery service. This enables efficient menu planning based on the user's emotional state and selection and ordering of necessary ingredients.

[1166] A "terminal device" is an electronic device used to transmit ingredient information entered by the user to a server.

[1167] A "server means" is a central processing unit that receives ingredient information transmitted from terminal means and stores it in a database.

[1168] The "menu generation means" is a function in which the server means generates a weekly menu based on stored ingredient information.

[1169] The "method for listing missing ingredients" is a function that lists any missing ingredients needed for the menu generated by the menu generation method.

[1170] "Display means" refers to a function that displays to the user the list of missing ingredients compiled by the missing ingredient list means, as well as the weekly menu.

[1171] "Emotion recognition means" refers to a function that recognizes the user's emotional state using an emotion engine provided in the server.

[1172] The "suggestion mechanism" is a function that makes suggestions based on the user's emotional state as recognized by the emotion recognition mechanism.

[1173] The "ordering method" refers to the function of ordering the ingredients suggested by the suggestion method through a delivery service.

[1174] System Configuration

[1175] To realize this invention, a system including the following hardware and software is required.

[1176] 1. Terminal device: This is a device for the user to input information about the food items currently in the refrigerator, and is implemented through a smartphone or tablet application.

[1177] 2. Server method: Receive ingredient information sent from the terminal and store it in a database. This can be done using a general cloud server or a local server.

[1178] 3. Database: Manages stored ingredient and recipe information, using SQL or NoSQL databases.

[1179] 4. Menu generation method: This is a server-based function that generates weekly menus based on ingredient information stored in a database and receiving feedback from the emotion engine.

[1180] 5. Ingredient Listing Method: This is a server-based function that compares all the ingredients required for the menu generated by the menu generation method with the ingredient information currently held by the user, and lists any missing ingredients.

[1181] 6. Display means: A terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[1182] 7. Emotion Recognition Means: An emotion engine is used to analyze the user's emotions, and the results are fed back to the menu generation means.

[1183] 8. Proposed method: Based on the user's emotional information obtained by the emotion recognition method, the system provides the user with the most suitable meal suggestions.

[1184] 9. Ordering method: This is a function that allows users to order the suggested ingredients through a delivery service.

[1185] Usage example

[1186] 1. User input and submission of ingredient information:

[1187] The user uses a smartphone app to input information about the ingredients they currently possess, such as "salmon," "chicken," "cabbage," and "carrots." The device then sends this information to the server.

[1188] 2. Preservation of ingredient information:

[1189] The server receives ingredient information sent from the terminal and stores it in the database. This allows for centralized management of the ingredient information held by users.

[1190] 3. Analysis of emotional data:

[1191] While the user is using the app, the emotion engine analyzes the user's facial expressions and voice to recognize their current emotional state. Specifically, it detects emotions such as "tired," "stressed," and "happy."

[1192] 4. Menu generation:

[1193] Based on the stored ingredient information and feedback from the emotion engine, the server selects appropriate recipes from the recipe database and generates a weekly meal plan. For example, if the user is tired, it will suggest a meal to replenish their energy.

[1194] 5. List of missing ingredients:

[1195] All the ingredients needed for the menu are identified, and by comparing them with the user's current ingredient list, any missing ingredients are highlighted. For example, "soy sauce" or "chicken broth."

[1196] 6. Displaying the results:

[1197] The generated weekly meal plan and a list of missing ingredients are displayed to the user via a smartphone app. For example, it might say, "Monday: Grilled salmon and vegetables to replenish energy," or "Tuesday: Relaxing chicken dish."

[1198] 7. Ordering missing ingredients:

[1199] Based on the displayed list of missing ingredients, users can order the necessary ingredients through a delivery service.

[1200] Examples of specific cases and prompt statements

[1201] Specific example:

[1202] The user enters "salmon" and "cabbage" on their smartphone.

[1203] The terminal sends ingredient information to the server.

[1204] The server saves ingredient information to the database.

[1205] The emotion engine recognizes from the camera footage that the user is "tired."

[1206] The app suggests "grilled salmon and stir-fried vegetables" as a suitable meal for nutritional supplementation.

[1207] The list of missing ingredients includes "soy sauce".

[1208] I ordered the soy sauce I was running low on through a delivery service.

[1209] Example of a prompt:

[1210] User-input ingredient information: ["Salmon", "Cabbage"]

[1211] User's emotional state: "Tired"

[1212] Suggested menu: "Grilled salmon and stir-fried vegetables"

[1213] List of missing ingredients: ["Soy sauce"]

[1214] Delivery order for missing ingredients: ["Soy sauce"]

[1215] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1216] Step 1:

[1217] The user uses a smartphone application to input information about the ingredients they currently possess. For example, they might input ingredients such as "salmon," "chicken," "cabbage," and "carrots." The entered ingredient information is then sent from the device to the server. The input is in text format, and this text data is sent to the server.

[1218] Step 2:

[1219] The server stores ingredient information sent from the terminal into an SQL database. The database stores and manages ingredient information along with user-specific identification information. This allows for centralized management of the ingredient information each user possesses.

[1220] Step 3:

[1221] While the user is using the application, the smartphone's camera and microphone are used to perform emotion recognition. The emotion engine analyzes the input audio and video data to recognize the user's emotional state. For example, it may recognize emotions such as "tired," "stressed," or "happy." This emotion information is sent to the server as text data.

[1222] Step 4:

[1223] The server selects an appropriate recipe from the recipe database based on emotional data sent from the emotion engine and ingredient information stored in the database. The menu generation system selects recipes based on different themes for each day of the week and automatically generates a week's worth of menus. The generated menus are saved in text format as a list of dish names and required ingredients.

[1224] Step 5:

[1225] The server lists all the necessary ingredients based on the menu generated by the menu generation system. Next, it compares the listed ingredients with the user's ingredient information stored in the database to identify any missing ingredients. For example, "soy sauce" and "chicken broth" might be listed as missing ingredients. This information is also saved as text data.

[1226] Step 6:

[1227] The server sends the generated weekly meal plan and list of missing ingredients to the user's device, displaying them visually. Users can check the meal plan and list of missing ingredients through a smartphone application. For example, specific meal plans such as "Monday: Grilled salmon and vegetables to replenish energy" and "Tuesday: Relaxing chicken dish" are displayed.

[1228] Step 7:

[1229] Users can order missing ingredients through a delivery service based on the displayed list of missing ingredients. The ordering system automatically confirms the order in conjunction with the delivery service's API and delivers the ingredients at the user's desired time. For example, if "Missing: Soy sauce" and "Missing: Chicken broth" are ordered through the delivery service, the items will be delivered.

[1230] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1231] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1232] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1233] [Fourth Embodiment]

[1234] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1235] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1236] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1237] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1238] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1239] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1240] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1241] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1242] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1243] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1244] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1245] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1246] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1247] overview

[1248] This invention is a system that automatically generates an optimal menu based on the user's ingredient information, lists any missing ingredients, and provides them to the user. This system enables users to efficiently plan meals and simplify grocery shopping.

[1249] System Configuration

[1250] This system includes the following components:

[1251] 1. Terminal: An electronic device used by the user to input ingredient information and send it to the server.

[1252] 2. Server: A central processing unit that receives ingredient information sent from terminals and stores it in a database.

[1253] 3. Database: A system for managing stored information on ingredients and recipes.

[1254] 4. Menu generation method: A server-based function that automatically generates a week's worth of menus based on stored ingredient information.

[1255] 5. Method for listing missing ingredients: A server-side function that compares all the ingredients required for the generated menu with the ingredients the user currently possesses and lists any missing ingredients.

[1256] 6. Display means: A terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[1257] Program processing

[1258] Enter and submit ingredient information

[1259] The user enters information about the food items currently in their refrigerator into the terminal. The terminal converts this information into a data structure and sends it to the server. For example, if the user has "salmon," "chicken," "cabbage," and "carrots," the terminal will send this data.

[1260] Food ingredient information storage

[1261] The server receives ingredient information sent from the terminal and stores it in the database along with the user's identification information. This allows for centralized management of the ingredient information held by each user.

[1262] Menu generation

[1263] The server selects appropriate recipes from its recipe database based on saved ingredient information. It searches for recipes based on different themes for each day of the week (e.g., fish dishes on Mondays, meat dishes on Tuesdays) and generates a week's worth of meal plans.

[1264] List of missing ingredients

[1265] The server identifies all the ingredients needed for the generated weekly menu and compares them to the ingredients the user currently possesses. Based on the comparison, it lists any missing ingredients.

[1266] Displaying Results

[1267] The terminal displays the user a weekly menu and a list of missing ingredients received from the server. This allows users to plan their meals and makes it easier to buy the necessary ingredients.

[1268] Specific example

[1269] User: Uses a smartphone to input "salmon," "chicken," "cabbage," and "carrots."

[1270] Terminal: Sends the entered ingredient information to the server.

[1271] Server: Receives ingredient information and saves it to the database.

[1272] Server: Generates a weekly menu. For example, recipes like "Grilled Salmon and Vegetables" might be selected for Monday, and "Stir-fried Chicken" for Tuesday.

[1273] Server: Lists the missing ingredients. For example, it might find that "soy sauce" and "chicken broth" are missing.

[1274] Terminal: Displays the generated weekly menu and a list of missing ingredients to the user.

[1275] In this way, the system efficiently supports users in their meal planning and shopping.

[1276] The following describes the processing flow.

[1277] Step 1:

[1278] The user enters the ingredients currently in their refrigerator into the terminal. For example, they might enter "salmon," "chicken," "cabbage," and "carrots."

[1279] Step 2:

[1280] The terminal sends the entered ingredient information to the server. At this time, the ingredient information, along with the user ID, is converted into a data structure such as JSON format.

[1281] Step 3:

[1282] The server receives ingredient information sent from the terminal. The received data is stored in a database using the user ID as the key.

[1283] Step 4:

[1284] The server uses the stored ingredient information from the database to search for appropriate recipes in the recipe database and generate a weekly meal plan. Recipes are selected based on different themes for each day of the week (e.g., fish dishes on Monday, meat dishes on Tuesday).

[1285] Step 5:

[1286] The server identifies all the ingredients needed for the generated weekly menu. This list is constructed based on recipe information stored in the recipe database.

[1287] Step 6:

[1288] The server compares the list of required ingredients with the ingredients the user currently possesses. Based on the comparison results, it lists the ingredients that are missing.

[1289] Step 7:

[1290] The terminal displays the weekly menu and a list of missing ingredients received from the server to the user. The display is in a visually easy-to-understand format. For example, it might show menus such as "Monday: Grilled salmon and vegetables" and "Tuesday: Stir-fried chicken," along with missing ingredients such as "Soy sauce, chicken broth."

[1291] (Example 1)

[1292] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1293] In today's busy lifestyle, it's not easy for users to efficiently plan meals and shop for groceries. Planning daily meals and checking for missing ingredients is time-consuming and laborious, making it a struggle for many. Therefore, there is a need for a system that automatically generates optimal meal plans based on the user's existing ingredient information and lists any missing ingredients.

[1294] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1295] In this invention, the server includes an electronic device means for transmitting ingredient information entered by the user to the server, a central processing unit means for receiving the ingredient information transmitted from the electronic device means and storing it in a data structure, a menu generation function that automatically generates a weekly menu based on the stored ingredient information, a comparison calculation means for listing any missing ingredients required for the menu generated by the menu generation function, and a visual display means for displaying the list of missing ingredients and the weekly menu listed by the comparison calculation means to the user. As a result, the user can efficiently plan a weekly menu, easily identify any missing ingredients, and create a shopping list.

[1296] "Ingredient information" refers to information about ingredients that the user possesses, including details such as the name of the ingredient, quantity, and expiration date.

[1297] "Electronic device means" refers to a device used by the user to input ingredient information and transmit that information to a server, and is an electronic device such as a smartphone or tablet.

[1298] The "central processing unit" is a server that has the function of receiving food ingredient information transmitted from the electronic device and storing it in a database.

[1299] "Data structure" refers to the format and structure of the data that the server uses to store the transmitted ingredient information.

[1300] The "menu generation function" is a server-based feature that automatically generates a week's worth of menus based on saved ingredient information.

[1301] The "comparison calculation means" is a server-based processing function that compares all the ingredients required for the generated menu with the ingredients the user currently possesses, and identifies any missing ingredients.

[1302] "Visual display means" refers to a terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[1303] 1. Overview

[1304] This invention is a system that automatically generates an optimal menu based on the user's available ingredients and lists any missing ingredients, providing them to the user. This system allows users to efficiently plan meals and simplify grocery shopping.

[1305] 2. System Configuration

[1306] This system includes the following components:

[1307] 1. Terminal: An electronic device (e.g., smartphone) used by the user to input ingredient information and send it to the server.

[1308] 2. Server: A central processing unit that receives ingredient information sent from terminals and stores it in a data structure.

[1309] 3. Database: A system for managing stored information on ingredients and recipes.

[1310] 4. Menu generation function: A server-based function that automatically generates a week's worth of menus based on saved ingredient information.

[1311] 5. Comparison calculation means: A function that compares all the ingredients required for the generated menu with the ingredients the user currently possesses and lists the missing ingredients.

[1312] 6. Visual display means: A terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[1313] 3. Enter and submit ingredient information

[1314] The user enters information about the food items in their refrigerator at home into a device such as a smartphone. For example, if the user has "salmon," "chicken," "cabbage," and "carrots," they enter the names of these food items using a dedicated application on their device. The device converts the entered food information into JSON data format and sends it to the server using the HTTPS protocol.

[1315] 4. Preservation of ingredient information

[1316] The server receives JSON data sent from the terminal. The received ingredient information is stored in the database along with the user's identification information. If new ingredient information duplicates existing information, the data is kept up-to-date by updating the quantity and expiration date.

[1317] 5. Menu creation

[1318] The server searches for appropriate recipes from the recipe database based on ingredient information stored in the database. It filters recipes based on themes for each day of the week (e.g., fish dishes on Mondays, meat dishes on Tuesdays) and generates a week's worth of meal plans. This generation process uses a recipe selection algorithm based on a generative AI model.

[1319] 6. List of missing ingredients

[1320] The server creates a list of all necessary ingredients based on the generated weekly menu. It compares this list to the user's current ingredient list and lists any missing ingredients, including the quantity and type of each ingredient. For example, it might list ingredients the user doesn't have, such as soy sauce or chicken broth.

[1321] 7. Displaying the results

[1322] The server sends the generated weekly meal plan and a list of missing ingredients to the device in JSON format. The device parses the received JSON data and displays it visually to the user. Specifically, it displays the weekly meal plan and the list of missing ingredients in the application's UI.

[1323] 8. Specific Examples

[1324] User: Use a smartphone app to input "salmon," "chicken," "cabbage," and "carrots."

[1325] Terminal: Converts the entered information into JSON data and sends it to the server.

[1326] Server: Receives data and saves it to the database.

[1327] Server: Based on the saved data, it selects recipes such as "Grilled Salmon and Vegetables" and "Stir-fried Chicken" from the recipe database and generates a week's worth of meal plans.

[1328] Server: Based on the generated menu, it lists any missing ingredients such as "soy sauce" or "chicken broth".

[1329] Terminal: Displays the generated menu and a list of missing ingredients to the user.

[1330] Examples of input prompts for a generative AI model

[1331] The following is an example of a prompt to input into the generating AI model:

[1332] "Please create a one-week menu using salmon, chicken, cabbage, and carrots. Also, please list any ingredients that are missing from the menu."

[1333] In this way, the system efficiently supports users in their meal planning and shopping.

[1334] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1335] Step 1:

[1336] Enter and submit ingredient information

[1337] Users enter information about the ingredients they have in their refrigerator at home into a dedicated application on their smartphone.

[1338] Input: The user enters "salmon", "chicken", "cabbage", and "carrots".

[1339] The terminal converts the entered ingredient information into JSON data format. This data includes details such as the name of the ingredient, quantity, and expiration date.

[1340] Data processing: Converting ingredient information to JSON data format.

[1341] Output: Converted JSON data.

[1342] The terminal sends the converted JSON data to the server using the HTTPS protocol.

[1343] Function: Generate and send JSON data.

[1344] Step 2:

[1345] Food ingredient information storage

[1346] The server receives JSON data sent from the terminal. This is done along with the user's identification information.

[1347] Input: JSON data received via the HTTPS protocol.

[1348] The server parses the received JSON data and stores information about each ingredient as a separate entry in the database.

[1349] Data processing: Parsing JSON data and saving it to a database.

[1350] Output: Results of saving the analyzed food ingredient information to the database.

[1351] Function: Saves ingredient information.

[1352] Step 3:

[1353] Menu generation

[1354] The server retrieves all the ingredient information the user possesses from the database. This means that it retrieves the necessary data based on the stored ingredient information.

[1355] Input: Ingredient information stored in the database.

[1356] The server filters recipes from the recipe database based on themes for each day of the week and generates a week's worth of meal plans.

[1357] Data processing: Recipe search and filtering based on ingredient information.

[1358] Output: A weekly meal plan has been generated.

[1359] Function: Recipe selection and menu generation using a generative AI model.

[1360] Step 4:

[1361] List of missing ingredients

[1362] The server creates a list of all necessary ingredients based on the generated weekly menu.

[1363] Input: A generated weekly meal plan.

[1364] The server compares the user's current ingredient list with the generated ingredient list and lists any missing ingredients.

[1365] Data processing: Compare the list of ingredients you have with the list of ingredients you need.

[1366] Output: A list of missing ingredients.

[1367] Operation: Lists missing ingredients through comparison operations.

[1368] Step 5:

[1369] Displaying Results

[1370] The server sends the generated weekly meal plan and a list of missing ingredients to the terminal in JSON format.

[1371] Input: JSON data containing a weekly meal plan and a list of missing ingredients.

[1372] The device parses the received JSON data and displays it visually to the user through the application's UI.

[1373] Data processing: Parsing JSON data and converting it to a display format.

[1374] Output: Weekly meal plan and list of missing ingredients in a displayable format.

[1375] Users can view menus and lists of missing ingredients through their device screen to plan meals in an organized manner.

[1376] Function: Visual display and user confirmation of analyzed data.

[1377] (Application Example 1)

[1378] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1379] Traditional food information management systems generate menus based on the user's existing food information and list missing ingredients. However, they cannot notify users of missing ingredients in real time while shopping, making efficient shopping difficult. As a result, users may overlook necessary ingredients when shopping in physical stores. Furthermore, many systems lack features to guide users to the location of desired ingredients, contributing to prolonged shopping times. Consequently, planned shopping becomes difficult, leading to the problem of unnecessary purchases.

[1380] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1381] In this invention, the server includes terminal means for transmitting ingredient information entered by the user to the server, server means for receiving ingredient information transmitted from the terminal means and storing it in a database, menu generation means for generating a weekly menu based on the stored ingredient information, ingredient shortage list means for listing any missing ingredients needed for the menu generated by the menu generation means, display means for displaying the ingredient shortage list and the weekly menu listed by the ingredient shortage list means to the user, and real-time shopping support means for notifying the user of any missing ingredients in real time when they visit a store and supporting their purchase. As a result, the user can grasp in real time any missing ingredients while shopping at a physical store, enabling them to shop efficiently without missing any necessary ingredients.

[1382] A "terminal device" is an electronic device used to transmit ingredient information entered by the user to a server.

[1383] The "server means" is a central processing unit that receives ingredient information transmitted from the terminal means and stores it in a database.

[1384] The "menu generation means" is a function that automatically generates a weekly menu based on the stored ingredient information of the server means.

[1385] The "means for listing missing ingredients" is a function that lists any missing ingredients needed for the menu generated by the menu generation means.

[1386] "Display means" refers to a device for visually displaying to the user the list of insufficient ingredients and the weekly menu, which have been listed by the insufficient ingredient listing means.

[1387] A "real-time shopping support system" is a function that notifies users in real time of any missing ingredients when they visit a store, and supports their purchase.

[1388] A "database" is a collection of data used to manage information about ingredients and recipes.

[1389] "Comparing" means comparing the ingredients the user currently has with all the ingredients needed for the weekly meal plan to identify any missing ingredients.

[1390] overview

[1391] This invention is a system that automatically generates an optimal menu based on the user's ingredient information, lists any missing ingredients, and provides them to the user. This system enables users to efficiently plan meals and shop for groceries easily. Furthermore, real-time shopping support in stores allows users to purchase necessary ingredients efficiently without missing any.

[1392] System Configuration

[1393] This system includes the following components:

[1394] 1. Terminal device: An electronic device used by the user to input ingredient information and send it to the server. A smartphone is a specific example.

[1395] 2. Server: A central processing unit that receives ingredient information transmitted from terminals and stores it in a database.

[1396] 3. Database: A system for managing stored ingredient information and recipe information. Database management systems such as SQLite are used.

[1397] 4. Menu Generation Method: This is a server-based function that automatically generates a week's worth of menus based on stored ingredient information. It uses a generation AI model that takes into account user preferences and allergy information.

[1398] 5. Method for listing missing ingredients: This is a server-side function that compares all the ingredients required for the generated menu with the ingredients the user currently possesses and lists any missing ingredients.

[1399] 6. Display means: A terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[1400] 7. Real-time shopping support: This function notifies users in real time of any missing ingredients when they visit a store, and supports their purchase.

[1401] Program processing

[1402] Enter and submit ingredient information

[1403] The user uses their smartphone to input information about the food items currently in their refrigerator. This includes barcode scanning and manual entry. The entered food information is converted into a data structure and sent to the server.

[1404] Food ingredient information storage

[1405] The server receives ingredient information sent from the terminal and stores it in the database along with the user's identification information. This allows for centralized management of the ingredient information held by each user.

[1406] Menu generation

[1407] The server selects appropriate recipes from its recipe database based on stored ingredient information. It searches for recipes based on different themes for each day of the week (e.g., fish dishes on Mondays, meat dishes on Tuesdays) and generates a week's worth of meal plans. A generation AI model is used to take user preferences and allergy information into consideration.

[1408] List of missing ingredients

[1409] The server identifies all the ingredients needed for the generated weekly menu and compares them to the ingredients the user currently possesses. Based on the comparison, it lists any missing ingredients.

[1410] Displaying Results

[1411] The terminal displays the user a weekly menu and a list of missing ingredients received from the server. This allows users to plan their meals and makes it easier to buy the necessary ingredients.

[1412] Real-time shopping support

[1413] When users visit a store, they can use their smartphones to check in real time which ingredients are running low. Furthermore, the system can use in-store location information to guide them to the shelf where their desired ingredients are located.

[1414] Specific example

[1415] User: Uses a smartphone to input "chicken," "grilled salmon," "cabbage," and "carrots."

[1416] Server: Receives ingredient information and saves it to the database.

[1417] Server: Uses a generation AI model to generate a weekly menu, such as "Grilled salmon and vegetables" for Monday and "Stir-fried chicken" for Tuesday.

[1418] Server: Lists the missing ingredients. For example, it might find that "onions" and "soy sauce" are missing.

[1419] Display method: The generated weekly menu and a list of missing ingredients are displayed to the user.

[1420] Real-time shopping support: Displays in real time which ingredients are missing at the store the user is visiting, and guides the user to the shelf location using location information.

[1421] Example of a prompt

[1422] "Please list the ingredients the user currently possesses. Next, list all the ingredients needed for the generated weekly meal plan, and then compare and list any missing ingredients."

[1423] In this way, the system efficiently supports users in their meal planning and shopping.

[1424] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1425] Step 1: Enter and submit ingredient information.

[1426] Users input information about ingredients they have at home using their smartphones. Input methods include barcode scanning and manual entry. Specifically, users can scan the barcodes on ingredients using their smartphone's camera or enter text within the app. The entered information is converted into a data structure and sent to the server in JSON format. <Input: Ingredient information (e.g., chicken, cabbage) / Output: Ingredient data in JSON format>

[1427] Step 2: Save ingredient information

[1428] The server receives ingredient information sent from the terminal and stores it in the database along with the user's identification information. The database used here is SQLite. Specifically, the server parses the received JSON data, converts each ingredient's information into the appropriate format, and inserts it into the database. <Input: Ingredient data in JSON format / Output: Ingredient information stored in the database>

[1429] Step 3: Menu Generation

[1430] The server selects appropriate recipes from a recipe database based on ingredient information stored in the database. It uses a generative AI model to generate a week's worth of menus based on a daily theme (e.g., fish dishes on Mondays, meat dishes on Tuesdays). Specifically, the server selects the most suitable recipes considering the user's preferences and allergy information to create a weekly menu. <Input: Ingredient information from the database / Output: Weekly menu list>

[1431] Step 4: List the missing ingredients

[1432] The server identifies all the ingredients needed for the generated weekly menu and compares them with the ingredients the user currently possesses in the database. Based on this comparison, it lists the missing ingredients. Specifically, it matches the list of ingredients included in the menu with the user's ingredient database and extracts only the missing items. <Input: Weekly menu list, ingredient information from the database / Output: List of missing ingredients>

[1433] Step 5: Displaying the results

[1434] The device visually displays a weekly meal plan and a list of missing ingredients, sent from the server, to the user. Specifically, the smartphone app analyzes the received data, formats it for easy viewing on the user interface, and displays it. <Input: Weekly meal plan, list of missing ingredients / Output: Information displayed on the device screen>

[1435] Step 6: Real-time shopping support

[1436] When a user visits a store, they can use their smartphone to check for missing ingredients in real time. This includes a feature that uses in-store location information to guide them to the location of ingredients. Specifically, it uses GPS and in-store beacons to determine the user's current location and displays the optimal route to the shelves where each ingredient is located. <Input: User location information, missing ingredient list / Output: Real-time ingredient location guidance>

[1437] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1438] overview

[1439] This invention combines a system that automatically generates an optimal menu based on the user's ingredient information and lists any missing ingredients with an emotion engine that recognizes the user's emotions. The emotion engine analyzes the user's emotional state and provides a menu based on that state. This system allows users to efficiently plan meals and enjoy meals that suit their emotions.

[1440] System Configuration

[1441] This system includes the following components:

[1442] 1. Terminal: An electronic device used by the user to input ingredient information and send it to the server.

[1443] 2. Server: A central processing unit that receives ingredient information sent from terminals and stores it in a database.

[1444] 3. Database: A system for managing stored information on ingredients and recipes.

[1445] 4. Menu generation method: A server-based function that automatically generates a week's worth of menus based on saved ingredient information and feedback from an emotion engine.

[1446] 5. Method for listing missing ingredients: A server-side function that compares all the ingredients required for the generated menu with the ingredients the user currently possesses and lists any missing ingredients.

[1447] 6. Display means: A terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[1448] 7. Emotion Engine: A system that analyzes the user's emotions and feeds the results back into the menu generation process.

[1449] Program processing

[1450] Enter and submit ingredient information

[1451] The user enters information about the ingredients currently in their refrigerator into the terminal. For example, they might enter "salmon," "chicken," "cabbage," and "carrots." The terminal converts this information into a data structure and sends it to the server.

[1452] Food ingredient information storage

[1453] The server receives ingredient information sent from the terminal and stores it in a database along with the user's identification information. This allows for centralized management of the ingredient information held by each user.

[1454] User emotion recognition

[1455] While the user is using the device, the emotion engine analyzes the user's input and voice to recognize their current emotional state. For example, it can detect emotions such as "tired," "stressed," or "happy."

[1456] Menu generation

[1457] The server uses ingredient information stored in the database and receives feedback from the emotion engine. It then selects appropriate recipes from the recipe database to generate a weekly meal plan. Recipes are selected based on different themes for each day of the week (e.g., fish dishes on Monday, meat dishes on Tuesday), and dishes that reflect the user's emotional state are suggested.

[1458] List of missing ingredients

[1459] The server identifies all the ingredients needed for the generated weekly menu. This list is constructed based on recipe information stored in the recipe database. The system compares the list of required ingredients with the ingredients the user currently possesses, and lists any missing ingredients.

[1460] Displaying Results

[1461] The device displays the user a weekly menu and a list of missing ingredients received from the server, along with suggestions based on the user's emotional state. For example, it might display menus such as "Monday: Grilled salmon and vegetables to replenish energy" and "Tuesday: Relaxing chicken dish," along with a note saying, "Missing: Soy sauce, chicken broth."

[1462] Specific example

[1463] User: Uses a smartphone to input "salmon," "chicken," "cabbage," and "carrots."

[1464] Terminal: Sends the entered ingredient information to the server.

[1465] Server: Receives ingredient information and saves it to the database.

[1466] Emotion Engine: Recognizes that the user is "tired" from their voice input and feeds that information back to the server.

[1467] Server: Based on feedback from the emotion engine, it selects recipes with a refreshing effect and generates a weekly meal plan. For example, Monday might be a "refreshing salad," Tuesday a "relaxing soup," and so on.

[1468] Server: List the missing ingredients and identify the missing ingredients as "soy sauce" and "chicken broth".

[1469] Terminal: Visually displays the generated weekly menu and a list of missing ingredients to the user.

[1470] This system efficiently plans meals while reflecting the user's emotional state and identifies missing ingredients, thereby enriching the user's life.

[1471] The following describes the processing flow.

[1472] Step 1:

[1473] The user enters the ingredients currently in their refrigerator into the terminal. For example, they might enter "salmon," "chicken," "cabbage," and "carrots."

[1474] Step 2:

[1475] The terminal sends the entered ingredient information to the server. At this time, the ingredient information, along with the user ID, is converted into a data structure such as JSON format.

[1476] Step 3:

[1477] The server receives ingredient information sent from the terminal. The received data is stored in a database using the user ID as the key.

[1478] Step 4:

[1479] While the user is using the device, the emotion engine analyzes the user's input and voice. For example, it can recognize emotions such as "tired," "stressed," or "happy."

[1480] Step 5:

[1481] The emotion engine sends the recognized user's emotional state to the server. For example, the data is sent in the format "User ID: user123, Emotional State: Tired".

[1482] Step 6:

[1483] The server receives feedback from the emotion engine based on stored ingredient information in the database. It then selects appropriate recipes from the recipe database based on different themes for each day of the week, generating a weekly menu. For example, a "refreshing recipe" might be selected for Monday, and a "relaxing recipe" for Tuesday.

[1484] Step 7:

[1485] The server generates a list of all the ingredients needed for the generated week's menu. This is done based on recipe information stored in the recipe database.

[1486] Step 8:

[1487] The server compares the created list of required ingredients with the ingredient information the user possesses. Based on the comparison results, it lists the ingredients that are missing.

[1488] Step 9:

[1489] The terminal displays the user a weekly menu and a list of missing ingredients received from the server. For example, the menu might be displayed in the format of "Monday: Grilled salmon and vegetables to replenish energy" and "Tuesday: Relaxing chicken dish," with "soy sauce" and "chicken broth" being indicated as missing ingredients.

[1490] (Example 2)

[1491] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1492] In today's busy lifestyle, there is a need to efficiently manage ingredients and plan meals, but current systems have difficulty suggesting meals that take into account the user's emotional state. As a result, users may not be able to eat meals that suit their specific emotional state, which could lead to decreased satisfaction with meals. This invention aims to achieve more personalized meal suggestions and necessary ingredient management by taking the user's emotional state into consideration.

[1493] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1494] In this invention, the server includes terminal means for transmitting ingredient information entered by the user to the server, server means for receiving ingredient information transmitted from the terminal means and storing it in a database, menu generation means for generating a weekly menu based on the ingredient information stored by the server means, emotion analysis means connected to the server means for analyzing the user's emotional state and providing feedback thereto to the menu generation means, ingredient deficiency listing means for listing any missing ingredients needed for the menu generated by the menu generation means, and display means for displaying the ingredient deficiency list and the weekly menu listed by the ingredient deficiency listing means to the user. This makes it possible to suggest personalized menus that take into account the user's emotional state and to list any missing ingredients.

[1495] A "terminal device" is an electronic device used to transmit ingredient information entered by the user to a server.

[1496] A "server means" is a central processing unit that receives ingredient information transmitted from terminal means and stores it in a database.

[1497] A "menu generation means" is a means that has the function of generating a weekly menu based on the ingredient information stored by the server means.

[1498] The "emotion analysis means" is a system connected to a server means that analyzes the user's emotional state and provides feedback to the menu generation means.

[1499] The "method for listing missing ingredients" is a method for listing the missing ingredients needed for the menu generated by the menu generation method.

[1500] "Display means" refers to the terminal's function for displaying to the user the list of missing ingredients compiled by the missing ingredient list means, as well as the weekly menu.

[1501] Modes for carrying out the invention

[1502] This invention combines a system that automatically generates an optimal menu based on ingredient information entered by the user and lists any missing ingredients with an emotion analysis means that recognizes the user's emotions. The following describes in detail how this system works.

[1503] System Components

[1504] This system includes the following components:

[1505] 1. Terminal device: An electronic device used by the user to input ingredient information and send it to the server. Examples include smartphones, tablets, and personal computers.

[1506] 2. Server: A central processing unit that receives ingredient information and stores it in a database. The server uses an SQL database to store ingredient information.

[1507] 3. Database: A system for managing stored ingredient information and recipe information. Examples include database management systems such as MySQL and PostgreSQL.

[1508] 4. Menu generation method: A server-based function that automatically generates a week's worth of menus based on stored ingredient information and feedback from an emotion analysis method.

[1509] 5. Emotion Analysis System: A system that analyzes the user's emotional state and provides the results to the menu generation system. Natural language processing technology is used for emotion analysis.

[1510] 6. Method for listing missing ingredients: A server-side function that compares all the ingredients required for the generated menu with the ingredients the user currently possesses and lists any missing ingredients.

[1511] 7. Display method: A terminal function that visually displays the generated weekly menu and list of missing ingredients to the user. This may be done using a smartphone app or a web browser.

[1512] System operation

[1513] 1. The user enters information about the ingredients in their refrigerator into the device. Specifically, they open the smartphone app and enter ingredients such as "salmon," "chicken," "cabbage," and "carrots."

[1514] 2. The terminal converts this input information into a data structure (e.g., JSON format) and sends it to the server. The transmission uses an HTTP POST request.

[1515] 3. The server analyzes the received data and stores it in the database along with the user's identification information. This allows for centralized management of ingredient information for each user.

[1516] 4. When a user inputs an emotional state such as "I'm tired today" via voice input, the terminal analyzes this voice and sends it to an emotion analysis system. The emotion analysis system uses natural language processing technology to analyze this information and provides feedback on the emotional state, such as "tired."

[1517] 5. Based on ingredient information and feedback from emotion analysis, the server selects appropriate recipes from the recipe database and generates a weekly menu. The menu consists of different themes for each day of the week, and dishes are suggested based on the user's emotional state.

[1518] 6. The server lists all the ingredients needed for the week's menu and compares this to the ingredients the user currently possesses. Missing items are listed, and items such as "soy sauce" and "chicken broth" are identified.

[1519] 7. The device visually displays the generated weekly meal plan and a list of missing ingredients. For example, it might say, "Monday: Grilled salmon and vegetables to replenish energy," or "Tuesday: Relaxing chicken dish."

[1520] Examples of specific cases and prompt statements

[1521] Example: A user inputs "salmon," "chicken," "cabbage," and "carrots" from their refrigerator using their smartphone, and enters "tired" as their current emotional state. The system then selects recipes with a refreshing effect and generates a week's worth of meal plans.

[1522] Example of a prompt:

[1523] "I have salmon, chicken, cabbage, and carrots in my refrigerator. I'm also very tired right now. Based on that, please suggest a meal plan for the week and tell me what ingredients I'm missing."

[1524] This system allows users to easily create optimal menus that reflect their emotional state and automatically identify any missing ingredients.

[1525] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1526] Step 1:

[1527] The user enters food information on their device. The user opens the app on their smartphone and enters the food items they have in their refrigerator, such as "salmon," "chicken," "cabbage," and "carrots."

[1528] Input: The name of the ingredient entered by the user.

[1529] Output: The input data is saved to the terminal.

[1530] Specific action: The user enters the name of the ingredient into a text input field in the smartphone app and presses the "Submit" button.

[1531] Step 2:

[1532] The terminal converts the entered ingredient information into a data structure and sends it to the server. The terminal converts the entered text information into JSON format and sends an HTTP POST request to the server.

[1533] Input: The name of the ingredient entered by the user.

[1534] Output: Ingredient information sent to the server (in JSON format).

[1535] Specific operation: The program converts the ingredient information entered on the terminal into JSON format and sends an HTTP POST request to the URL endpoint.

[1536] Step 3:

[1537] The server receives the submitted ingredient information and stores it in the database. The server receives an HTTP request, parses the JSON data, and stores it in the SQL database.

[1538] Input: Ingredient information (in JSON format) sent from the device.

[1539] Output: Ingredient information stored in the database.

[1540] Specific operation: The server receives ingredient information using a REST API, parses it, and executes an INSERT query to insert it into the database along with the user's identification information, and then saves it.

[1541] Step 4:

[1542] The user enters their emotional state. They use their device to voice-input "I'm tired today."

[1543] Input: User's emotional state.

[1544] Output: The input data is saved to the terminal.

[1545] Specific operation: The user uses the voice input function to communicate their emotional state to the device.

[1546] Step 5:

[1547] The device transmits its emotional state to the emotion analysis system. The emotion analysis system uses natural language processing to analyze this information and provides feedback on the emotional state, such as "fatigue."

[1548] Input: User's emotional state.

[1549] Output: Emotional data analyzed by the emotion analysis tool.

[1550] Specific operation: The device converts the audio data into text data, which is then sent to the sentiment analysis engine. The analysis engine uses a natural language processing model to identify the emotional state (e.g., tired) and returns the result.

[1551] Step 6:

[1552] The server generates a menu based on ingredient information and emotional feedback from emotion analysis tools. The server retrieves ingredient information from a database, selects appropriate recipes based on the feedback, and generates a weekly menu.

[1553] Input: Food ingredient information obtained from a database, emotional feedback from emotion analysis tools.

[1554] Output: A week's worth of menus.

[1555] Specific operation: The server retrieves stored ingredient information using SQL queries, selects a suitable recipe from the recipe database considering the sentiment analysis results, and creates a weekly meal plan. For example, for a "tired" state, it selects a recipe that provides energy.

[1556] Step 7:

[1557] The server generates a weekly menu and creates a list of necessary ingredients. It then compares this list with the user's existing ingredient information and lists any missing ingredients.

[1558] Input: Weekly menu, user ingredient information stored in the database.

[1559] Output: List of missing ingredients.

[1560] Specific operation: The server identifies all ingredients included in the week's menu, compares them with the current user's ingredient information using an SQL query, and lists any missing ingredients.

[1561] Step 8:

[1562] The terminal displays the generated weekly menu and a list of missing ingredients to the user. It receives response data from the server and displays it on the user interface.

[1563] Input: Weekly menu and list of missing ingredients from the server.

[1564] Output: Weekly meal plan and list of missing ingredients displayed to the user.

[1565] Specific operation: The device receives a response from the server and displays menus such as "Monday: Grilled salmon and vegetables to replenish energy" and "Tuesday: Relaxing chicken dish" through the application's UI, as well as a list of missing ingredients such as "Missing: Soy sauce, chicken broth".

[1566] (Application Example 2)

[1567] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1568] In modern households, food management and meal planning are not done efficiently, making it particularly difficult to suggest appropriate meals that suit the user's emotional state. Furthermore, the inability to select the optimal ingredients leads to food waste.

[1569] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes terminal means for transmitting ingredient information entered by the user to the server, server means for receiving the ingredient information transmitted from the terminal means and storing it in a database, menu generation means for generating a weekly menu based on the ingredient information stored by the server means, ingredient deficiency listing means for listing any missing ingredients needed for the menu generated by the menu generation means, display means for displaying the ingredient deficiency list and the weekly menu listed by the ingredient deficiency listing means to the user, emotion recognition means for recognizing the user's emotional state using an emotion engine provided in the server means, suggestion means for making suggestions based on the user's emotional state recognized by the emotion recognition means, and order means for ordering the ingredients suggested by the suggestion means through a delivery service. This enables efficient menu planning based on the user's emotional state and selection and ordering of necessary ingredients.

[1570] A "terminal device" is an electronic device used to transmit ingredient information entered by the user to a server.

[1571] A "server means" is a central processing unit that receives ingredient information transmitted from terminal means and stores it in a database.

[1572] The "menu generation means" is a function in which the server means generates a weekly menu based on stored ingredient information.

[1573] The "method for listing missing ingredients" is a function that lists any missing ingredients needed for the menu generated by the menu generation method.

[1574] "Display means" refers to a function that displays to the user the list of missing ingredients compiled by the missing ingredient list means, as well as the weekly menu.

[1575] "Emotion recognition means" refers to a function that recognizes the user's emotional state using an emotion engine provided in the server.

[1576] The "suggestion mechanism" is a function that makes suggestions based on the user's emotional state as recognized by the emotion recognition mechanism.

[1577] The "ordering method" refers to the function of ordering the ingredients suggested by the suggestion method through a delivery service.

[1578] System Configuration

[1579] To realize this invention, a system including the following hardware and software is required.

[1580] 1. Terminal device: This is a device for the user to input information about the food items currently in the refrigerator, and is implemented through a smartphone or tablet application.

[1581] 2. Server method: Receive ingredient information sent from the terminal and store it in a database. This can be done using a general cloud server or a local server.

[1582] 3. Database: Manages stored ingredient and recipe information, using SQL or NoSQL databases.

[1583] 4. Menu generation method: This is a server-based function that generates weekly menus based on ingredient information stored in a database and receiving feedback from the emotion engine.

[1584] 5. Ingredient Listing Method: This is a server-based function that compares all the ingredients required for the menu generated by the menu generation method with the ingredient information currently held by the user, and lists any missing ingredients.

[1585] 6. Display means: A terminal function that visually displays the generated weekly menu and list of missing ingredients to the user.

[1586] 7. Emotion Recognition Means: An emotion engine is used to analyze the user's emotions, and the results are fed back to the menu generation means.

[1587] 8. Proposed method: Based on the user's emotional information obtained by the emotion recognition method, the system provides the user with the most suitable meal suggestions.

[1588] 9. Ordering method: This is a function that allows users to order the suggested ingredients through a delivery service.

[1589] Usage example

[1590] 1. User input and submission of ingredient information:

[1591] The user uses a smartphone app to input information about the ingredients they currently possess, such as "salmon," "chicken," "cabbage," and "carrots." The device then sends this information to the server.

[1592] 2. Preservation of ingredient information:

[1593] The server receives ingredient information sent from the terminal and stores it in the database. This allows for centralized management of the ingredient information held by users.

[1594] 3. Analysis of emotional data:

[1595] While the user is using the app, the emotion engine analyzes the user's facial expressions and voice to recognize their current emotional state. Specifically, it detects emotions such as "tired," "stressed," and "happy."

[1596] 4. Menu generation:

[1597] Based on the stored ingredient information and feedback from the emotion engine, the server selects appropriate recipes from the recipe database and generates a weekly meal plan. For example, if the user is tired, it will suggest a meal to replenish their energy.

[1598] 5. List of missing ingredients:

[1599] All the ingredients needed for the menu are identified, and by comparing them with the user's current ingredient list, any missing ingredients are highlighted. For example, "soy sauce" or "chicken broth."

[1600] 6. Displaying the results:

[1601] The generated weekly meal plan and a list of missing ingredients are displayed to the user via a smartphone app. For example, it might say, "Monday: Grilled salmon and vegetables to replenish energy," or "Tuesday: Relaxing chicken dish."

[1602] 7. Ordering missing ingredients:

[1603] Based on the displayed list of missing ingredients, users can order the necessary ingredients through a delivery service.

[1604] Examples of specific cases and prompt statements

[1605] Specific example:

[1606] The user enters "salmon" and "cabbage" on their smartphone.

[1607] The terminal sends ingredient information to the server.

[1608] The server saves ingredient information to the database.

[1609] The emotion engine recognizes from the camera footage that the user is "tired."

[1610] The app suggests "grilled salmon and stir-fried vegetables" as a suitable meal for nutritional supplementation.

[1611] The list of missing ingredients includes "soy sauce".

[1612] I ordered the soy sauce I was running low on through a delivery service.

[1613] Example of a prompt:

[1614] User-input ingredient information: ["Salmon", "Cabbage"]

[1615] User's emotional state: "Tired"

[1616] Suggested menu: "Grilled salmon and stir-fried vegetables"

[1617] List of missing ingredients: ["Soy sauce"]

[1618] Delivery order for missing ingredients: ["Soy sauce"]

[1619] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1620] Step 1:

[1621] The user uses a smartphone application to input information about the ingredients they currently possess. For example, they might input ingredients such as "salmon," "chicken," "cabbage," and "carrots." The entered ingredient information is then sent from the device to the server. The input is in text format, and this text data is sent to the server.

[1622] Step 2:

[1623] The server stores ingredient information sent from the terminal into an SQL database. The database stores and manages ingredient information along with user-specific identification information. This allows for centralized management of the ingredient information each user possesses.

[1624] Step 3:

[1625] While the user is using the application, the smartphone's camera and microphone are used to perform emotion recognition. The emotion engine analyzes the input audio and video data to recognize the user's emotional state. For example, it may recognize emotions such as "tired," "stressed," or "happy." This emotion information is sent to the server as text data.

[1626] Step 4:

[1627] The server selects an appropriate recipe from the recipe database based on emotional data sent from the emotion engine and ingredient information stored in the database. The menu generation system selects recipes based on different themes for each day of the week and automatically generates a week's worth of menus. The generated menus are saved in text format as a list of dish names and required ingredients.

[1628] Step 5:

[1629] The server lists all the necessary ingredients based on the menu generated by the menu generation system. Next, it compares the listed ingredients with the user's ingredient information stored in the database to identify any missing ingredients. For example, "soy sauce" and "chicken broth" might be listed as missing ingredients. This information is also saved as text data.

[1630] Step 6:

[1631] The server sends the generated weekly meal plan and list of missing ingredients to the user's device, displaying them visually. Users can check the meal plan and list of missing ingredients through a smartphone application. For example, specific meal plans such as "Monday: Grilled salmon and vegetables to replenish energy" and "Tuesday: Relaxing chicken dish" are displayed.

[1632] Step 7:

[1633] Users can order missing ingredients through a delivery service based on the displayed list of missing ingredients. The ordering system automatically confirms the order in conjunction with the delivery service's API and delivers the ingredients at the user's desired time. For example, if "Missing: Soy sauce" and "Missing: Chicken broth" are ordered through the delivery service, the items will be delivered.

[1634] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1635] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1636] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1637] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1638] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1639] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1640] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1641] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1642] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1643] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1644] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1645] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1646] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1647] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1648] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1649] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1650] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1651] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1652] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1653] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1654] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1655] The following is further disclosed regarding the embodiments described above.

[1656] (Claim 1)

[1657] A terminal device that transmits ingredient information entered by the user to a server,

[1658] A server means that receives ingredient information transmitted from the terminal means and stores it in a database,

[1659] The aforementioned server means generates a weekly menu based on the stored ingredient information,

[1660] A means for listing missing ingredients necessary for the menu generated by the menu generation means,

[1661] A display means for displaying to the user the list of insufficient ingredients listed by the aforementioned insufficient ingredient listing means and the weekly menu,

[1662] A system that includes this.

[1663] (Claim 2)

[1664] The menu generation means selects recipes based on different types of ingredients for each day of the week, according to claim 1.

[1665] (Claim 3)

[1666] The system according to claim 1, wherein the means for listing missing ingredients lists missing ingredients by comparing the ingredients the user currently has with all the ingredients needed for the weekly menu.

[1667] "Example 1"

[1668] (Claim 1)

[1669] An electronic device means that transmits ingredient information entered by the user to a server,

[1670] A central processing unit that receives ingredient information transmitted from the aforementioned electronic device and stores it in a data structure,

[1671] The aforementioned central processing unit has a menu generation function that automatically generates a menu for one week based on the stored ingredient information,

[1672] A comparison calculation means that lists the missing ingredients required for the menu generated by the menu generation function,

[1673] A visual display means that displays to the user the list of insufficient ingredients and weekly menu listed by the comparison calculation means,

[1674] A system that includes this.

[1675] (Claim 2)

[1676] The menu generation function selects recipes based on different types of ingredients for each day of the week, according to claim 1.

[1677] (Claim 3)

[1678] The system according to claim 1, wherein the comparison calculation means lists any missing ingredients by comparing the ingredients the user currently possesses with all the ingredients needed for the weekly menu.

[1679] "Application Example 1"

[1680] (Claim 1)

[1681] A terminal device that transmits ingredient information entered by the user to a server,

[1682] A server means that receives ingredient information transmitted from the terminal means and stores it in a database,

[1683] The aforementioned server means generates a weekly menu based on the stored ingredient information,

[1684] A means for listing missing ingredients necessary for the menu generated by the menu generation means,

[1685] A display means for displaying to the user the list of insufficient ingredients listed by the aforementioned insufficient ingredient listing means and the weekly menu,

[1686] A real-time shopping support system that notifies the user in real time of any missing ingredients when they visit the store and supports their purchase,

[1687] A system that includes this.

[1688] (Claim 2)

[1689] The menu generation means selects recipes based on different types of ingredients for each day of the week, according to claim 1.

[1690] (Claim 3)

[1691] The system according to claim 1, wherein the means for listing missing ingredients lists missing ingredients by comparing the ingredients the user currently has with all the ingredients needed for the weekly menu.

[1692] "Example 2 of combining an emotion engine"

[1693] (Claim 1)

[1694] A terminal device that transmits ingredient information entered by the user to a server,

[1695] A server means that receives ingredient information transmitted from the terminal means and stores it in a database,

[1696] The aforementioned server means generates a weekly menu based on the stored ingredient information,

[1697] An emotion analysis means connected to the server means, which analyzes the user's emotional state and provides feedback thereto to the menu generation means,

[1698] A means for listing missing ingredients necessary for the menu generated by the menu generation means,

[1699] A display means for displaying to the user the list of insufficient ingredients listed by the aforementioned insufficient ingredient listing means and the weekly menu,

[1700] A system that includes this.

[1701] (Claim 2)

[1702] The system according to claim 1, wherein the menu generation means selects a recipe based on the user's emotional state.

[1703] (Claim 3)

[1704] The system according to claim 1, wherein the means for listing missing ingredients lists missing ingredients by comparing the ingredients the user currently has with all the ingredients needed for the weekly menu.

[1705] "Application example 2 when combining with an emotional engine"

[1706] (Claim 1)

[1707] A terminal device that transmits ingredient information entered by the user to a server,

[1708] A server means that receives ingredient information transmitted from the terminal means and stores it in a database,

[1709] The aforementioned server means generates a weekly menu based on the stored ingredient information,

[1710] A means for listing missing ingredients necessary for the menu generated by the menu generation means,

[1711] A display means for displaying to the user the list of insufficient ingredients listed by the aforementioned insufficient ingredient listing means and the weekly menu,

[1712] An emotion recognition means that recognizes the user's emotional state using an emotion engine provided in the server means,

[1713] A suggestion means that makes suggestions based on the user's emotional state recognized by the emotion recognition means,

[1714] An ordering means for ordering the ingredients proposed by the aforementioned proposed means through a delivery service,

[1715] A system that includes this.

[1716] (Claim 2)

[1717] The menu generation means selects recipes based on different types of ingredients for each day of the week, according to claim 1.

[1718] (Claim 3)

[1719] The system according to claim 1, wherein the means for listing missing ingredients lists missing ingredients by comparing the ingredients the user currently has with all the ingredients needed for the weekly menu. [Explanation of symbols]

[1720] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A terminal device that transmits ingredient information entered by the user to a server, A server means that receives ingredient information transmitted from the terminal means and stores it in a database, The aforementioned server means generates a weekly menu based on the stored ingredient information, A means for listing missing ingredients necessary for the menu generated by the menu generation means, A display means for displaying to the user the list of insufficient ingredients listed by the aforementioned insufficient ingredient listing means and the weekly menu, A system that includes this.

2. The menu generation means selects recipes based on different types of ingredients for each day of the week, according to claim 1.

3. The system according to claim 1, wherein the means for listing missing ingredients lists missing ingredients by comparing the ingredients the user currently has with all the ingredients needed for the weekly menu.

Images