system

Abstract

We provide the system. [Solution] A means for the user to input a monthly budget, number of people, and foods to be excluded, A means of saving the entered data, A means of generating a menu that does not include excluded foods within the budget, based on stored data, A means for generating a list of necessary ingredients based on the generated menu, A method for adding items to a cart via an e-commerce site based on a list of ingredients and scheduling delivery on a specified date, A means of tracking expenses to ensure that monthly spending stays within budget based on saved data, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern dual-income households, it is difficult to find the time to plan daily menus. Also, grocery shopping is time-consuming, and it is even more difficult to manage to stay within the budget, especially. Furthermore, if there are foods to be excluded, it becomes even more difficult to plan menus while reflecting such requests. As a result, food waste and budget overruns are likely to occur. There is a need for an efficient system and method to solve these problems.

Means for Solving the Problems

[0005] To solve the above problems, the present invention provides the following means: a means for the user to input a monthly budget, number of people, and foods to be excluded, and a means for storing that data. Furthermore, a means for automatically generating a menu that does not include excluded foods and stays within the budget, based on the stored data. This also includes a means for generating a list of necessary ingredients based on the generated menu. Furthermore, a means for adding products to a cart via an e-commerce site and scheduling delivery on a specified date, based on the ingredient list. It also includes a means for tracking expenses to ensure that monthly spending stays within the budget, based on the stored data. This enables the user to manage their meals efficiently and within budget without any hassle.

[0006] "User" refers to an individual or group of people who use this system to manage their household meals.

[0007] "Monthly budget" refers to the amount of money a user can spend on groceries and related products in a month.

[0008] "Number of people" refers to the number of people who will be eating the meal for which the system will suggest a menu.

[0009] "Foods to exclude" refers to specific ingredients or food groups that users do not want to include in their meals for health or personal reasons.

[0010] "Means of input" refers to interfaces or devices that allow users to provide data such as budget, number of people, and excluded foods to the system.

[0011] "Means of storage" refers to the function of retaining entered data in a database or other storage device.

[0012] "Methods for generating menus" refers to a function that automatically creates daily menus that meet specific conditions using saved data and a recipe database.

[0013] "Means for generating ingredient lists" refers to a function that creates a list of necessary ingredients based on the generated menu.

[0014] An "e-commerce site" refers to an online marketplace where goods and services are bought and sold via the internet.

[0015] "Adding to cart" refers to the function of adding desired products to a shopping cart using the system of an e-commerce site.

[0016] "Means of scheduling delivery" refers to a function that plans delivery so that purchased goods are delivered to the user at the date and time requested.

[0017] "Means of tracking expenses" refers to features that monitor and manage purchase history and expenses in real time to ensure that users' spending stays within budget.

[0018] "Menu generation method" refers to a function that selects and suggests the most suitable dishes to the user, taking into account the seasonality of ingredients and nutritional balance.

[0019] "Interface" refers to the screens and operating methods that allow a user to interact with the system, and includes means for adding, deleting, and modifying elements. [Brief explanation of the drawing]

[0020] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5]It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying Out the Invention

[0021] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described according to the accompanying drawings.

[0022] First, the language used in the following description will be explained.

[0023] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0024] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0025] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0026] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0028] [First Embodiment]

[0029] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0030] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0033] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0036] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0040] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0041] This invention is a system for streamlining household meal management, particularly for dual-income households, automating the process of planning daily menus and purchasing and delivering ingredients within a budget. The system allows users to input their monthly budget, number of people, and foods to be excluded, and then generates daily menus and lists of necessary ingredients. Furthermore, it provides a series of functions for purchasing and delivering ingredients in conjunction with e-commerce sites.

[0042] Program execution steps

[0043] 1. User registration:

[0044] User: Open the dedicated application or web interface and enter your monthly budget, family size, and foods you want to exclude. Specifically, the user might enter information such as "budget of 30,000 yen, family of 4, exclude nuts."

[0045] Terminal: Locally checks the entered data to ensure there are no formatting issues. After the check is complete, the data is sent to the server.

[0046] Server: Receives data sent from terminals, stores it in the database, and manages user-specific settings.

[0047] 2. Menu suggestions:

[0048] Server: Retrieves user configuration information from the database and generates a menu that does not include excluded foods within the budget. For example, it calculates the daily cost for four people with a budget of 30,000 yen and uses an algorithm to select recipes that do not include nuts.

[0049] Terminal: Displays menu suggestions sent from the server to the user. For example, it might display "Dinner on day 1: Chicken curry, Day 2: Stir-fried vegetables."

[0050] 3. Generating the ingredient list:

[0051] Server: Based on the proposed menu, it generates a list of necessary ingredients. For example, it creates a list such as, "For chicken curry, you need chicken, onions, curry powder, spices, etc."

[0052] Terminal: Provides an interface that displays a list of ingredients to the user, allowing the user to review and make necessary corrections.

[0053] 4. Grocery purchase and delivery settings:

[0054] Server: Based on the confirmed list of ingredients, the server connects with the API of the e-commerce site to add ingredients to the cart and schedule the delivery date and time. For example, using the Yahoo! Shopping API, it might set "Purchase chicken, onions, and curry powder on [Month] [Day], and deliver on [Month] [Day]."

[0055] Terminal: Displays a screen to the user to notify them of purchase details and delivery date / time, and for the user to confirm.

[0056] 5. Budget Management:

[0057] Server: Tracks expenses in real time based on purchase history. Checks actual spending against the user's budget and manages it to stay within budget. For example, if spending is likely to exceed the budget near the end of the month, it suggests lower-cost alternative recipes.

[0058] Terminal: Display budget progress and warning messages to the user, allowing them to constantly monitor the budget status.

[0059] Specific example

[0060] For example, a user registers the following conditions:

[0061] Monthly budget: 30,000 yen

[0062] Family size: 4 people

[0063] Excluded items: Nuts

[0064] Based on these conditions, the system performs the following steps: First, at the beginning of the month, it generates a 30-day menu that fits the conditions. For example, a one-week menu might be suggested as follows:

[0065] Day 1: Lunch: Sandwich, Dinner: Chicken Curry

[0066] Day 2: Lunch: Spaghetti, Dinner: Stir-fried vegetables

[0067] ...

[0068] The system then creates a list of ingredients needed for each menu item. For example, it might present a list such as, "For Day 1's chicken curry, you'll need chicken, onions, curry powder, spices, etc." The user reviews this list, makes any necessary adjustments, places an order using the e-commerce site's API, and schedules delivery for the specified day. This entire process allows users to easily decide on their daily menus and automatically purchase the necessary ingredients. The system also manages spending in real time to stay within budget, ensuring a stable diet even amid rising prices.

[0069] The following describes the processing flow.

[0070] Step 1: The user opens the dedicated application or web interface, enters their monthly budget, number of family members, and foods they wish to exclude, and presses the submit button.

[0071] Step 2: The terminal locally validates the entered data and performs error checks. If there are no problems with validation, the data is sent to the server.

[0072] Step 3: The server receives the data sent from the terminal and saves it to the database. For example, it saves information such as "budget of 30,000 yen, family of 4, nuts excluded."

[0073] Step 4: The server retrieves the user's settings information from the database and generates daily menus based on it. It selects menus that do not include excluded foods within the budget and saves them on the server.

[0074] Step 5: The server sends the generated menu data to the terminal and displays it to the user. For example, it might display something like, "Dinner on day 1 is chicken curry, and on day 2 it's stir-fried vegetables."

[0075] Step 6: The user reviews the menu displayed on the device and adds or removes ingredients or adjusts quantities as needed.

[0076] Step 7: The device sends the final ingredient list, which has been reviewed and corrected by the user, to the server.

[0077] Step 8: Based on the confirmed ingredient list, the server adds items to the cart via the e-commerce site's API. Add the necessary ingredients to the cart one by one, and confirm the purchase once all items are available.

[0078] Step 9: After the server completes the purchase process, it schedules the delivery date and time. The schedule takes into account the user's preferred delivery date.

[0079] Step 10: The server sends the delivery details and order contents to the terminal and displays a confirmation screen to the user. The user reviews the delivery date and time and purchase details and makes any necessary changes.

[0080] Step 11: The server tracks expenses in real time based on purchase history. Check the budget and make any necessary adjustments to prevent exceeding the budget.

[0081] Step 12: The device displays budget progress and warning messages to the user. For example, it provides information such as, "Current spending is 15,000 yen, remaining budget is 15,000 yen."

[0082] This allows users to efficiently manage grocery shopping and delivery, and plan their daily meals within their budget.

[0083] (Example 1)

[0084] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0085] Currently, for busy households, such as dual-income families, deciding on daily menus and purchasing and managing ingredients within budget is a significant burden. Furthermore, there is a need for efficient methods of purchasing ingredients while considering budget management, seasonality of ingredients, and nutritional balance. Conventional systems have found it difficult to automate and optimize all of these elements, leaving a considerable amount of effort to the user.

[0086] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0087] In this invention, the server includes means for the user to input a monthly budget, number of people, and foods to be excluded; means for storing the input data; means for generating a menu that does not include excluded foods and stays within the budget based on the stored data; means for generating a list of necessary ingredients based on the generated menu; means for adding products to a cart via an e-commerce site based on the ingredient list and scheduling delivery on a specified date; means for tracking expenses to ensure that monthly expenditures stay within budget based on the stored data; means for checking the format of the input content in real time and transmitting the data if the format is determined to be correct; and means for generating low-cost alternative recipes based on expense tracking. This makes it possible for the user to easily decide on daily menus and automatically purchase and manage ingredients within budget.

[0088] A "user" is someone who uses the system to input monthly budgets, the number of people, and foods to be excluded, and then instructs the system to generate daily menus and lists of necessary ingredients.

[0089] The "means of input" refer to a system that provides an interface for users to input their monthly budget, number of people, and foods they wish to exclude.

[0090] "Means of saving" refers to a system that saves entered data to a storage device such as a database so that it can be used later.

[0091] "Methods for generating menus" refer to algorithms or programs that automatically generate menus within a budget and excluding excluded foods, based on stored data.

[0092] "Means for generating ingredient lists" refers to algorithms or programs that create a list of necessary ingredients based on the generated menu.

[0093] An "e-commerce site" is a platform for selling and buying goods online.

[0094] The "add to cart" method is a system that automates the purchase process of products on e-commerce sites based on a list of ingredients.

[0095] "Means of scheduling delivery" refers to a time management system for delivering products purchased on an e-commerce site to users at a specified date and time.

[0096] A "means of tracking expenses" is a system that monitors and manages expenses in real time based on stored data to ensure that monthly spending stays within budget.

[0097] "A means of checking the format of input content in real time" refers to a system that checks immediately whether there are any problems with the format of the data entered by the user.

[0098] "Means for generating low-cost alternative recipes" are algorithms or programs that, based on cost tracking results, suggest alternative menus that can be prepared at a lower cost if the budget is likely to be exceeded.

[0099] This invention is a system designed to streamline meal management in busy households, such as dual-income families. Specifically, it provides a process for determining daily menus and automating the purchase and delivery of ingredients within a budget.

[0100] This system's program primarily functions through the collaboration of three entities: servers, terminals, and users. The following describes the processing procedures and the hardware and software used for each entity.

[0101] User actions

[0102] Users enter their monthly budget, family size, and foods they wish to exclude through a dedicated application or web interface. For example, a user might enter "Budget: 30,000 yen, Family size: 4, Exclude nuts." The entered data is checked in real time on the terminal and sent to the server only after it is confirmed to be in the correct format.

[0103] Server Processing

[0104] The server performs the following main tasks:

[0105] 1. Data Storage and Management: Receive data submitted by users and store it in a database. For example, use a database system such as MongoDB or MySQL®.

[0106] 2. Menu Generation: Based on saved data, menus are generated that stay within budget and do not include excluded foods. Specifically, a generation AI model is used to algorithmically select menus that meet the user's criteria.

[0107] 3. Generating an ingredient list: Based on the suggested menu, generate a list of necessary ingredients. For example, create a list such as, "For chicken curry, you'll need chicken, onions, curry powder, spices, etc."

[0108] 4. Integration with e-commerce sites: Based on the confirmed list of ingredients, the system integrates with the API of an e-commerce site (e.g., Yahoo! Shopping API) to add ingredients to the cart and schedule delivery dates and times.

[0109] 5. Budget Management: Track expenses in real time based on purchase history and check actual spending against the user's budget. If the budget is likely to be exceeded, generate lower-cost alternative recipes.

[0110] Terminal processing

[0111] The terminal (the device operated by the user) provides the following functions:

[0112] 1. Data Verification and Transmission: The data entered by the user is verified in real time, and if there are no formatting issues, it is sent to the server.

[0113] 2. Display of Menus and Ingredient Lists: Menu suggestions and ingredient lists received from the server are displayed in the user interface. For example, it might display "Dinner on Day 1: Chicken Curry, Day 2: Stir-fried Vegetables."

[0114] 3. User Confirmation and Modification: Provide an interface that allows users to review the ingredient list and delivery details and make modifications as needed.

[0115] 4. Displaying budget progress: Display budget progress and warning messages to allow users to always check the budget status.

[0116] Specific example

[0117] For example, a user registers the following conditions:

[0118] Monthly budget: 30,000 yen

[0119] Family size: 4 people

[0120] Excluded items: Nuts

[0121] Based on these conditions, the system performs the following steps: First, it generates a 30-day menu at the beginning of the month within the given conditions. For example, a one-week menu might be suggested as follows:

[0122] Day 1: Lunch: Sandwich, Dinner: Chicken Curry

[0123] Day 2: Lunch: Spaghetti, Dinner: Stir-fried vegetables

[0124] ...

[0125] The system then creates a list of ingredients needed for each menu item. For example, it might present a list such as, "For Day 1's chicken curry, you'll need chicken, onions, curry powder, spices, etc." The user reviews this list, makes any necessary adjustments, places an order using the e-commerce site's API, and schedules delivery for the specified day. This entire process allows users to easily decide on their daily menus and automatically purchase the necessary ingredients. The system also manages spending in real time to stay within budget, ensuring a stable diet even amid rising prices.

[0126] Example of a prompt

[0127] The following are specific examples of prompts to input into a generative AI model:

[0128] "Please create a system to manage meals for a dual-income household with a monthly budget of 30,000 yen for four people. Please provide detailed instructions, including specific menu suggestions when the user specifies nuts as a food to exclude, how to generate a list of necessary ingredients, and the process for purchasing and delivering those ingredients."

[0129] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0130] Program processing flow

[0131] Step 1: User Registration

[0132] input

[0133] User: Enter your budget, family size, and foods you want to exclude using the dedicated application or web interface.

[0134] Specific input example: "Budget 30,000 yen, family of 4, exclude nuts"

[0135] Specific actions

[0136] Terminal: Checks user input in real time. If there are formatting issues, displays an error message and prompts correction. If the format is correct, encrypts the data and sends it to the server.

[0137] output

[0138] The input data is sent to the server.

[0139] Step 2: Save Data

[0140] input

[0141] User data sent from the device

[0142] Specific actions

[0143] Server: Analyzes received data and adds it to the database as a new record. Simultaneously, it generates a user ID and stores it as user initial information. For example, it might use MongoDB or MySQL to manage the data.

[0144] output

[0145] User information is stored in the database.

[0146] Step 3: Menu Generation

[0147] input

[0148] User budget, family size, and exclusion food list retrieved from the database.

[0149] Specific actions

[0150] Server: Executes database queries to retrieve user information. Based on the retrieved data, a generative AI model is used to generate menus that stay within budget and exclude excluded foods. For example, it calculates the daily cost for four people with a monthly budget of 30,000 yen and uses an algorithm to select recipes that do not contain nuts.

[0151] output

[0152] The generated menu information is sent to the terminal.

[0153] Step 4: Menu Display

[0154] input

[0155] Menu information sent from the server

[0156] Specific actions

[0157] Terminal: Displays received menu information on the user interface. For example, it might suggest "Dinner on day 1 is chicken curry" or "Dinner on day 2 is stir-fried vegetables."

[0158] output

[0159] The user checks the menu.

[0160] Step 5: Generating the ingredient list

[0161] input

[0162] Proposed menu information

[0163] Specific actions

[0164] Server: Retrieves the necessary ingredients for each menu item from the database and creates a list. For example, it generates a list such as, "Chicken curry requires chicken, onions, curry powder, spices, etc."

[0165] output

[0166] The generated list of ingredients is sent to the terminal.

[0167] Step 6: Check and correct the ingredient list

[0168] input

[0169] Ingredient list sent from the server

[0170] Specific actions

[0171] Terminal: Displays the ingredient list in the user interface and provides buttons such as "Edit," "Delete," and "Add." When the user makes a modification, the data is sent back to the server.

[0172] output

[0173] Confirmed list of ingredients

[0174] Step 7: Purchase ingredients and set up delivery.

[0175] input

[0176] Confirmed list of ingredients

[0177] Specific actions

[0178] Server: Based on the ingredient list, it connects with the API of an e-commerce site (e.g., Yahoo! Shopping API). For example, it adds "chicken, onion, curry powder" to the cart and sets the delivery date to "Month Day".

[0179] output

[0180] The purchase process is completed on the e-commerce site, and delivery is scheduled.

[0181] Step 8: Notification of purchase details and delivery date / time

[0182] input

[0183] Purchase procedure and delivery schedule information

[0184] Specific actions

[0185] Terminal: The information is displayed as a pop-up message on the user interface, providing a screen for the user to confirm.

[0186] output

[0187] A user verification flag is returned to the server.

[0188] Step 9: Budget management and alternative recipe suggestions

[0189] input

[0190] Purchase history data

[0191] Specific actions

[0192] Server: Tracks expenses in real time and manages spending against the user's budget. If the budget is likely to be exceeded, it uses a generative AI model to suggest lower-cost alternative recipes.

[0193] output

[0194] An alternative recipe is sent to the device.

[0195] Step 10: Displaying budget progress

[0196] input

[0197] Budget progress and warning messages sent from the server

[0198] Specific actions

[0199] Terminal: The budget management screen displays information such as "Current expenses: 28,000 yen" and "Remaining budget: 2,000 yen," and a warning icon flashes to notify the user.

[0200] output

[0201] User verification and budget adjustment

[0202] (Application Example 1)

[0203] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0204] There are challenges in streamlining household meal management, such as automating daily meal planning, grocery purchasing, and delivery within budget for dual-income households. Furthermore, it's necessary to minimize the effort users have to manually check and adjust ingredients, and to provide real-time updates on budget progress and warning messages.

[0205] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0206] In this invention, the server includes means for the user to input a monthly budget, number of people, and foods to be excluded; means for storing the input data; means for generating a menu that does not include excluded foods and stays within the budget based on the stored data; means for generating a list of necessary ingredients based on the generated menu; means for adding products to an online shopping cart based on the list of necessary ingredients and scheduling delivery on a specified date; means for tracking expenses based on the stored data to ensure that monthly expenditures stay within the budget; and means for providing an interface that allows the user to check the budget progress and warning messages. This automates the daily tasks of deciding on menus, purchasing ingredients, and delivery, and enables more efficient budget management.

[0207] "A means for users to input monthly budgets, number of people, and foods to exclude" refers to an interface for inputting basic data for monthly meal planning, taking into account constraints such as family structure, budget, and allergies.

[0208] "Means of saving entered data" refers to a function that stores information entered by the user on a cloud server or database, making it accessible later.

[0209] "A means of generating a menu that stays within budget and excludes excluded foods based on saved data" refers to an algorithm that uses saved user data to automatically plan a menu that stays within budget and excludes specified foods.

[0210] "A means of generating a list of necessary ingredients based on a generated menu" refers to a function that creates a list of ingredients needed to prepare a meal according to a planned menu.

[0211] "A method for adding products to an online shopping cart based on a list of necessary ingredients and scheduling delivery on a specified date" refers to a function that uses the API of an online shopping site to add products to the cart based on the created ingredient list and arranges delivery at the date and time desired by the user.

[0212] "A means of tracking expenses to ensure monthly spending stays within budget based on saved data" refers to a feature that monitors purchase history and current spending in real time and adjusts spending to stay within budget, in order to help users manage their budget.

[0213] "Means of providing an interface that allows users to check budget progress and warning messages" refers to a function that provides a screen through an application or web browser that allows users to check budget progress and related warning messages in real time.

[0214] The "means for generating menus that also consider the seasonality and nutritional balance of ingredients, and further call external e-commerce site APIs for purchasing products" refers to an algorithm and function that generates menus that include seasonal ingredients and nutritional balance, and then executes the process of purchasing ingredients using APIs from external shopping sites.

[0215] "Means of providing an interface for users to view, add, delete, or adjust the quantity of ingredients in an ingredient list" refers to a function that provides an interactive screen for users to add or delete items and adjust quantities as needed, based on the ingredient list they have created.

[0216] "A means of inputting prompt text into a generative AI model to generate recipes and ingredient information" refers to a function that executes a process in which the user inputs specific conditions or preferences in text format, and the generative AI model generates the optimal recipe and related ingredient information.

[0217] The system of this invention provides a comprehensive approach to streamline family meal management. Specifically, it supports an interface in which the user inputs a monthly budget, the number of family members, and foods to be excluded, and stores and uses this information. Data entered via a device (such as a smartphone or PC) is stored in a cloud-based database.

[0218] The server generates menus that fit within the budget and do not include excluded foods, based on the stored data. For example, if the registered data is "monthly budget of 30,000 yen, family size of 4, excluded foods: nuts," the server will generate and recommend appropriate recipes that meet those conditions. Based on the generated menus, the server automatically generates a list of necessary ingredients. This ingredient list provides an interface that allows the user to review, add, delete, and adjust the quantities of ingredients.

[0219] Furthermore, based on the list of necessary ingredients, the server interacts with the API of an online shopping site (for example, a hypothetical e-commerce API), adds the items to the cart, and schedules delivery of the ingredients at a date and time specified by the user. This allows users to obtain the necessary ingredients without leaving their homes.

[0220] Based on the stored data, the server tracks monthly spending in real time and manages it to stay within budget. It also provides an interface where users can check budget progress and receive warning messages, ensuring that spending is always properly managed. For example, it will issue an alert if there is a possibility of exceeding the budget near the end of the month.

[0221] In addition, the server creates menus that take seasonality and nutritional balance into consideration, supporting users' health management. The generated list of ingredients automates the process of purchasing by calling APIs of external e-commerce sites. Users can easily perform these operations through the interface.

[0222] As a concrete example, here is an example of a prompt message that the user might enter:

[0223] "For the development of a food and beverage management application, please propose a 30-day menu based on the following input information: budget of 30,000 yen, family size of 4, and exclude nuts. Additionally, please generate a list of ingredients required for each menu."

[0224] The main hardware / software used in implementing this invention includes a cloud-based database, server-side programming languages ​​(e.g., Python and Flask), and libraries to facilitate communication with external APIs (e.g., Requests). This reduces user effort and enables an efficient meal management system.

[0225] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0226] Step 1:

[0227] Users enter their budget, family size, and foods they wish to exclude using an application or web interface. Once the data is entered, it is sent from the device to the server. The entered data includes monthly budget, family composition, and allergy information.

[0228] Input: Budget, number of family members, foods to exclude

[0229] Output: User data sent to the server

[0230] Step 2:

[0231] The server saves the received data to a database. Once stored in the database, the user's settings become available for subsequent processing.

[0232] Input: User data sent to the server

[0233] Output: User information stored in the database

[0234] Step 3:

[0235] The server generates a menu that fits within the budget and excludes excluded foods, based on stored user information. This process also applies an algorithm that takes into account the seasonality and nutritional balance of ingredients.

[0236] Input: User information stored in the database

[0237] Output: Generated menu

[0238] Step 4:

[0239] The server creates a list of necessary ingredients based on the generated menu. This list details the ingredients required for each menu recipe, including quantities and units.

[0240] Input: Generated menu

[0241] Output: List of required ingredients

[0242] Step 5:

[0243] The ingredient list created by the server is sent to the user's terminal, where the user can review it. The user can add, delete, or adjust the contents of the ingredient list through the interface. The user's actions are resent to the server.

[0244] Input: List of required ingredients

[0245] Output: User-reviewed and adjusted ingredient list

[0246] Step 6:

[0247] Based on the final list of ingredients, the server calls the e-commerce site's API to add the ingredients to the cart and schedule delivery for the date and time specified by the user. The results of the API calls are stored on the server and notified to the user.

[0248] Input: User-confirmed and adjusted ingredient list

[0249] Output: Purchase information and delivery schedule in e-commerce

[0250] Step 7:

[0251] The server tracks purchase history and current spending in real time, managing it to stay within budget. If the budget is likely to be exceeded, it notifies the user with alternatives or warnings.

[0252] Input: Purchase history, current spending data

[0253] Output: Spending tracking information, warning messages

[0254] Step 8:

[0255] Users can check budget progress and warning messages through the interface. The server provides this information in real time, allowing users to always be aware of their current budget status.

[0256] Input: Expense tracking information, warning messages

[0257] Output: Budget progress and warning messages displayed in the user interface.

[0258] An example of a specific prompt message is as follows:

[0259] "For the development of a food and beverage management application, please propose a 30-day menu based on the following input information: budget of 30,000 yen, family size of 4, and exclude nuts. Additionally, please generate a list of ingredients required for each menu."

[0260] The above is a description of the specific processing steps.

[0261] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0262] This invention is a system designed to streamline household meal management, particularly for dual-income households, automating the process of planning daily menus and purchasing and delivering ingredients within a budget. Furthermore, by incorporating an emotion engine that recognizes user emotions, this system provides a more personalized service by offering menu suggestions that take the user's emotional state into account.

[0263] Program execution steps

[0264] 1. User registration:

[0265] User: Open the dedicated application or web interface, enter your monthly budget, family size, and foods you want to exclude, and press the submit button. For example, enter "Budget 30,000 yen, family of 4, exclude nuts."

[0266] Terminal: Validates the entered data locally and performs error checks. If there are no problems, the data is sent to the server.

[0267] Server: Receives data sent from terminals and stores it in the database. Manages data based on user configuration information.

[0268] 2. Emotion recognition:

[0269] Terminal: When a user enters data, it collects facial expressions, voice tone, and text content.

[0270] Emotion Engine: Analyzes collected data to recognize the user's emotions. For example, it uses facial recognition and voice analysis technologies to extract emotions such as fatigue or stress.

[0271] Server: Receives the analysis results from the emotion engine and saves them to the database.

[0272] 3. Menu suggestions:

[0273] Server: Based on user configuration information and sentiment data, the server generates an optimal menu considering the user's emotional state. For example, if the server determines that the user is tired, it will suggest easy-to-prepare dishes.

[0274] Terminal: Displays menu suggestions sent from the server to the user. For example, it might suggest "Dinner on day 1: Chicken curry, Day 2: Stir-fried vegetables."

[0275] 4. Generating the ingredient list:

[0276] Server: Based on the proposed menu, it generates a list of necessary ingredients. For example, it creates a list such as, "For chicken curry, you need chicken, onions, curry powder, spices, etc."

[0277] Terminal: Provides an interface that displays a list of ingredients to the user, allowing the user to review and make necessary corrections.

[0278] 5. Grocery purchase and delivery settings:

[0279] Server: Based on the confirmed list of ingredients, the server adds items to the cart via the e-commerce site's API and schedules the delivery date and time. For example, it might set "Purchase chicken, onions, and curry powder on [Month] [Day], and deliver on [Month] [Day]."

[0280] Terminal: Displays a screen to the user to notify them of purchase details and delivery date / time, and for the user to confirm.

[0281] 6. Budget Management:

[0282] Server: Tracks expenses in real time based on purchase history. Checks actual spending against the user's budget and manages spending to stay within budget.

[0283] Terminal: Displays budget progress and warning messages to the user. For example, it might display "Current expenses are 15,000 yen, remaining budget is 15,000 yen."

[0284] Specific Example

[0285] For example, the user registers the following conditions:

[0286] Monthly budget: 30,000 yen

[0287] Number of family members: 4 people

[0288] Excluded items: Nuts

[0289] Based on these conditions, when the user makes an input today, the terminal collects the user's facial expressions and voice tones, and the emotion engine performs analysis. The emotion engine recognizes that the user is tired. Based on this information, the server proposes "Chicken curry for dinner on the first day and stir-fried vegetables on the second day" as a menu that can be easily cooked.

[0290] After that, the system creates a list of ingredients required for each menu. For example, it presents a list such as "For the chicken curry on the first day, chicken, onions, curry powder, spices...". After the user checks this list and makes adjustments if necessary, the user places an order using the API of the e-commerce site and schedules the delivery for the specified day.

[0291] Furthermore, the system manages to keep the expenditure within the budget in real time. For example, by displaying "The current expenditure is 15,000 yen and the remaining budget is 15,000 yen", the user can always check the budget status. Through this series of processes, the user can determine the daily menu without much effort and automatically purchase the necessary ingredients. Also, by combining the emotion engine, an optimal proposal according to the user's emotional state can be realized.

[0292] The following explains the processing flow.

[0293] Step 1: The user opens the dedicated application or web interface, enters their monthly budget, number of family members, and foods they want to exclude, and presses the submit button. For example, they might enter "Budget 30,000 yen, family of 4, exclude nuts."

[0294] Step 2: The terminal locally validates the entered data and performs error checks. If there are no errors, it sends the input data to the server.

[0295] Step 3: The server receives the data sent from the terminal and saves it to the database. This saves the user's configuration information for management purposes.

[0296] Step 4: The device collects the user's facial expressions, voice tone, and entered text. This data provides the foundational information needed to understand the user's emotional state.

[0297] Step 5: The emotion engine analyzes the collected emotion data to recognize the user's emotional state. For example, it uses facial recognition and voice analysis technologies to determine whether the user is tired, stressed, etc.

[0298] Step 6: The server receives the analysis results from the emotion engine and stores them in the database. This allows for tracking and long-term management of the user's emotional state.

[0299] Step 7: The server generates a menu based on the user's settings and emotional data. It also takes the user's emotional state into consideration; for example, if the user is tired, it selects dishes that are easy to prepare.

[0300] Step 8: The server sends the generated menu data to the terminal and displays it to the user. Here, it visually presents specific details such as "Dinner on day 1 is chicken curry, and on day 2 is stir-fried vegetables."

[0301] Step 9: The user checks the menu presented on the terminal and adds, deletes, or adjusts the quantity of ingredients as necessary. After confirmation, the user sends the modified content to the server.

[0302] Step 10: Based on the final ingredient list received from the user, the server uses the API of the e-commerce site to add the products to the cart. Each ingredient is sequentially added to the cart, and the purchase procedure is carried out.

[0303] Step 11: After the server completes the purchase procedure, it schedules the delivery date and time based on the user's preference. For example, set it as "Purchase chicken, onion, and curry powder on [date], deliver on [date]".

[0304] Step 12: The server sends the purchase details and delivery date and time to the terminal and displays a confirmation screen to the user. The user checks the order content and delivery schedule and makes changes if necessary.

[0305] Step 13: The server tracks the expenses in real-time based on the purchase history. It performs management to prevent budget overruns and proposes alternative recipes if there is a potential shortage.

[0306] Step 14: The terminal displays the budget progress and warning messages to the user. For example, provide information such as "The current expenditure is 15,000 yen, and the remaining budget is 15,000 yen" so that the user can always grasp the budget status.

[0307] In this way, the user can determine the daily menu without much effort and efficiently purchase and deliver the necessary ingredients. Also, by using the emotion engine, an optimal menu proposal according to the user's emotional state can be realized.

[0308] (Example 2)

[0309] Next, Example 2 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart device 14 is referred to as the "terminal".

[0310] This invention aims to reduce the burden on users, particularly in dual-income households, by automating the processes of daily meal planning, purchasing and delivery of ingredients within budget, and providing personalized services that take into account the user's emotional state. Furthermore, it aims to provide a system that integrates these processes, including meal suggestions that consider the season and nutritional balance, and budget management.

[0311] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0312] In this invention, the server includes means for storing user input data, means for analyzing emotional data, and means for generating menus and creating ingredient lists. This enables menu suggestions that take into account the user's emotional state, as well as automated ingredient purchasing and delivery.

[0313] "User input" refers to the process where users enter necessary information, such as monthly budgets, number of people, and foods they wish to exclude, through a dedicated application or web interface.

[0314] "Emotion recognition" is the process of recognizing a user's emotional state by analyzing their facial expressions, tone of voice, text content, and other factors.

[0315] "Menu generation" refers to the automatic creation of a menu that fits within the budget and does not exclude certain foods, based on the user's settings and sentiment data.

[0316] "Ingredient list generation" refers to creating a list of necessary ingredients based on the proposed menu.

[0317] An "e-commerce site" is an online platform for purchasing goods via the internet.

[0318] "Tracking" refers to the real-time management of expenses to ensure that a user's monthly spending stays within budget.

[0319] An "interface" is the user interface or operating environment through which a user interacts with a system, performing tasks such as inputting data, confirming information, and making corrections.

[0320] This invention is a system designed to streamline household meal management, particularly for dual-income households, automating the process of planning daily menus and purchasing and delivering ingredients within a budget. Furthermore, by incorporating an emotion engine that recognizes user emotions, this system provides a more personalized service by offering menu suggestions that take the user's emotional state into account.

[0321] User input and data storage

[0322] First, the user uses a dedicated application or web interface to enter their monthly budget, family size, and foods they wish to exclude. This data is validated and error-checked on the device before being sent to the server and stored in the database. For example, information such as "budget 30,000 yen, family of 4, exclude nuts" might be entered.

[0323] emotion recognition

[0324] When a user enters information, the device collects the user's facial expressions, voice tone, and text content. This collected data is analyzed by an emotion engine. Specifically, it uses facial recognition technology (e.g., Google® Cloud Vision) and voice analysis technology (e.g., IBM Watson®) to recognize the user's emotions. The analysis results are sent to a server and stored in a database.

[0325] Menu generation

[0326] The server generates an optimal menu based on saved user settings and emotional data. Considering the user's emotional state, for example, if the user is tired, a menu that is easy to prepare will be suggested. The suggested menu is sent to the terminal and displayed to the user. Specifically, a menu such as "Dinner on day 1: Chicken curry, Day 2: Stir-fried vegetables" might be suggested.

[0327] Generating and purchasing a list of ingredients.

[0328] The server generates a list of necessary ingredients based on the suggested menu. This list is displayed on the terminal for user confirmation and modification. The server then adds the items to the cart via the e-commerce site's API and schedules the delivery date and time. For example, it might be set to "Purchase chicken, onions, and curry powder on [Month] [Day], and deliver on [Month] [Day]."

[0329] Budget management and expense tracking

[0330] The server tracks purchase history in real time and checks actual spending against the user's budget. For example, it might display "Current spending: 15,000 yen, remaining budget: 15,000 yen." This allows users to constantly monitor their budget progress.

[0331] Specific example

[0332] For example, a user registers the following conditions:

[0333] Monthly budget: 30,000 yen

[0334] Family size: 4 people

[0335] Excluded items: Nuts

[0336] When you enter your information today, the device collects your facial expressions and voice tone, and an emotion engine analyzes this data. The emotion engine recognizes that the user is tired, and based on this information, the server suggests a menu that is easy to prepare. For example, it might suggest "Chicken curry for dinner on day 1, and stir-fried vegetables on day 2." A list of ingredients is also generated, for example, "For chicken curry on day 1, you'll need chicken, onions, curry powder, spices, etc." An order is then placed through an e-commerce site, and delivery is scheduled for the specified date. Finally, purchase history is tracked in real time, ensuring that your budget is properly managed.

[0337] Example of a prompt

[0338] "If a user has a monthly budget of 30,000 yen, a family of four, and wants to exclude nuts, and they are perceived as tired when deciding what to eat today, suggest chicken curry or stir-fried vegetables as the best menu for them."

[0339] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0340] Program execution steps

[0341] Step 1: User Registration

[0342] The user opens a dedicated application or web interface and enters their monthly budget, the number of family members, and the foods they want to exclude.

[0343] The terminal locally validates the input data and performs error checks. For example, it verifies that the budget and number of family members are in numerical format and that no invalid characters are included. If there are no problems, it sends the data to the server.

[0344] Input: User enters information such as "budget of 30,000 yen, family of 4, excludes nuts".

[0345] Processing: Local validation and error checking.

[0346] Output: The validated input data is sent to the server.

[0347] The server saves the received data to the database and returns a registration completion message to the user. If the entire process is successful, feedback such as "Registration Complete" is sent to the user.

[0348] Input: Validated data from the terminal.

[0349] Processing: Save data to the database.

[0350] Output: Send a registration completion message to the device.

[0351] Step 2: Emotion Recognition

[0352] The device collects facial expressions, voice tone, and text content when the user enters data.

[0353] Input: User's facial expression, voice tone, and input text.

[0354] Processing: Emotional data is collected by integrating facial expressions, voice tone, and text.

[0355] Output: Collected data is sent to the emotion engine.

[0356] The emotion engine analyzes collected data to recognize the user's emotions. Specifically, it uses facial recognition and voice analysis technologies to extract emotions such as fatigue and stress.

[0357] Input: Facial expressions, voice tone, and text data from the device.

[0358] Processing: Sentiment analysis using machine learning models.

[0359] Output: Send emotion recognition results to the server.

[0360] The server receives the analysis results from the emotion engine and stores them in the database.

[0361] Input: Analysis results from the emotion engine.

[0362] Processing: Save to database.

[0363] Output: Feedback of saved results.

[0364] Step 3: Menu Suggestions

[0365] The server generates an optimal menu based on the user's settings and emotional data. For example, if the server determines that the user is tired, it will suggest easy-to-prepare dishes.

[0366] Input: Configuration information and sentiment data.

[0367] Processing: Data calculations to generate menus by integrating configuration information and sentiment data.

[0368] Output: Sends the generated menu data to the terminal.

[0369] The terminal displays menu suggestions sent from the server to the user.

[0370] Input: Menu data from the server.

[0371] Processing: Reflecting menu data in the interface.

[0372] Output: Displays menus to the user, such as "Dinner on day 1 is chicken curry, and on day 2 is stir-fried vegetables."

[0373] Step 4: Generating the ingredient list

[0374] The server generates a list of necessary ingredients based on the proposed menu. For example, it might list "Chicken curry requires chicken, onions, curry powder, spices, etc."

[0375] Input: Suggested menu data.

[0376] Processing: Based on the menu data, retrieve a list of necessary ingredients from the database and create a list.

[0377] Output: Sends the generated ingredient list data to the terminal.

[0378] The terminal displays a list of ingredients to the user and provides an interface for the user to review and make necessary corrections.

[0379] Input: Ingredient list data from the server.

[0380] Processing: Reflecting ingredient list data in the interface.

[0381] Output: Displays a list of ingredients to the user, allowing for confirmation and modification.

[0382] Step 5: Purchase ingredients and set up delivery.

[0383] Based on the confirmed list of ingredients, the server adds items to the cart via the e-commerce site's API and schedules the delivery date and time.

[0384] Input: User-modified list of confirmed ingredients.

[0385] Processing: Add products to the cart and create a delivery schedule via the e-commerce site's API.

[0386] Output: Sends purchase and delivery details to the device.

[0387] The device notifies the user of the purchase details and delivery date and time, and displays a screen for the user to confirm.

[0388] Input: Purchase and delivery details data from the server.

[0389] Processing: Reflects purchase details and delivery schedule in the interface.

[0390] Output: Notify the user and display a confirmation screen.

[0391] Step 6: Budget Management

[0392] The server tracks expenses in real time based on purchase history. It checks actual spending against the user's budget and manages it to stay within budget.

[0393] Input: Purchase history data.

[0394] Processing: Real-time cost tracking and budget checking.

[0395] Output: Sends budget progress and warning messages to the terminal.

[0396] The terminal displays budget progress and warning messages to the user.

[0397] Input: Budget progress and warning messages from the server.

[0398] Processing: Reflect budget progress and warning messages in the interface.

[0399] Output: Displays budget progress and warning messages to the user.

[0400] (Application Example 2)

[0401] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0402] For dual-income households and busy modern people, deciding on daily menus is a significant burden, and purchasing ingredients within budget and cooking efficiently is a difficult challenge. Furthermore, failing to consider the user's emotional state during these tasks can increase stress and burden. In particular, the process of creating menus and purchasing ingredients while considering factors such as family size, food preferences, and allergy information is complex, and there is a need for an appropriate system to streamline these processes.

[0403] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for the user to input a monthly budget, number of people, and foods to be excluded; means for storing the input data; means for generating a menu that does not include excluded foods within the budget based on the stored data; means for generating a list of necessary ingredients based on the generated menu; means for adding products to a cart via an e-commerce site based on the ingredient list and scheduling delivery on a specified date; means for tracking expenses so that monthly expenditures stay within the budget based on the stored data; means for recognizing the user's emotions using facial expressions and voice tone; and means for suggesting a personalized menu based on the user's emotional data. This enables the suggestion of optimal menus with less stress according to the user's emotional state, as well as efficient purchase and management of ingredients.

[0404] A "user" is an individual or family member who uses the system.

[0405] A "budget" is the amount of money that is allowed to be spent on food and meals in a month.

[0406] "Number of people" refers to the number of family members or residents in a household who will be using the system for meal management.

[0407] "Foods to be excluded" are ingredients that users wish to avoid consuming for reasons such as health conditions, allergies, or personal preferences.

[0408] An "input method" is an interface that allows a user to provide the system with the information they need.

[0409] A "storage system" is a system that retains entered data and manages it in a way that allows it to be referenced when needed.

[0410] A "menu" is a plan that outlines the types and contents of meals a user will eat.

[0411] A "menu generation method" is a means of automatically creating a menu that does not exclude any foods, within the user's budget, based on the input data.

[0412] A "food ingredient list" is a list of ingredients needed based on the generated menu.

[0413] An "e-commerce site" is an online platform for purchasing goods via the internet.

[0414] A "delivery scheduling method" is a means of ensuring that food ingredients are delivered at the date and time specified by the user.

[0415] "Expense tracking methods" are means of monitoring and managing expenses to ensure that monthly expenditures stay within budget.

[0416] "Emotion recognition means" refers to technology that identifies emotions from a user's facial expressions and voice tone.

[0417] A "personalized menu" is an individually optimized meal plan that takes into account the user's emotional data.

[0418] This invention is a system that streamlines household meal management, automatically determining daily menus and purchasing and delivering ingredients within budget, especially for dual-income households. Furthermore, by incorporating an emotion engine that recognizes user emotions, it provides menu suggestions that take into account the user's emotional state, offering a more personalized service.

[0419] Hardware and software to be used

[0420] Smartphone: A device on which applications are installed.

[0421] Emotion recognition engine: "EmotionRecognition" analyzes facial expressions and "VoiceEmotionRecognition" analyzes voice tone.

[0422] Server: A server that maintains the database and provides APIs.

[0423] E-commerce API: An external API for purchasing groceries online.

[0424] Explanation of the process

[0425] 1. User registration:

[0426] Users input basic information such as their monthly budget, family size, and foods they wish to exclude using a dedicated application. This enables personalized meal planning and ingredient management for each household.

[0427] 2. Emotion recognition:

[0428] When a user enters information, the device collects facial expressions and voice tone, which are then analyzed by an emotion recognition engine. For example, facial recognition and voice analysis technologies can be used to extract emotional states such as whether the user is tired or stressed.

[0429] 3. Menu suggestions:

[0430] The server generates an optimal menu based on the user's settings and emotional data. For example, if it determines that the user is tired, it can suggest easy-to-prepare dishes.

[0431] 4. Generating the ingredient list:

[0432] Based on the suggested menu, a list of necessary ingredients is generated. This allows the user to efficiently prepare the required ingredients.

[0433] 5. Grocery purchase and delivery settings:

[0434] Based on the confirmed list of ingredients, the server adds items to the cart via the API of online e-commerce sites and schedules delivery dates and times. This allows users to obtain ingredients without any hassle.

[0435] 6. Budget Management:

[0436] The server tracks expenses based on purchase history to ensure they stay within budget and displays the user's current spending status. For example, it might show information such as "Current spending: ¥15,000, remaining budget: ¥15,000" in real time.

[0437] Specific example

[0438] For example, a user registers the following conditions:

[0439] Monthly budget: 30,000 yen

[0440] Family size: 4 people

[0441] Excluded items: Nuts

[0442] When a user enters information today, the device collects the user's facial expressions and voice tone, which are then analyzed by an emotion engine. The emotion engine recognizes that the user is tired. Based on this information, the server suggests easy-to-prepare meals, such as "Dinner on Day 1: Chicken Curry, Day 2: Stir-fried Vegetables." The system then creates a list of ingredients needed for each meal, presenting a list such as, "For Day 1's Chicken Curry: Chicken, onions, curry powder, spices..." The user reviews this list, makes any necessary adjustments, places an order using the e-commerce site's API, and schedules delivery for the specified day. Furthermore, the system manages spending in real time to keep it within budget, displaying information such as, "Current spending: 15,000 yen, remaining budget: 15,000 yen," allowing the user to constantly monitor their budget status.

[0443] Example of a prompt

[0444] User-entered data: Budget of 30,000 yen, family of 4, nuts excluded. Facial expressions and voice tone were analyzed to determine the user was tired. Based on this information, suggest a simple meal plan for the user: "Dinner on day 1: Chicken curry, Day 2: Stir-fried vegetables."

[0445] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0446] Step 1:

[0447] The user enters their monthly budget, number of people, and foods they want to exclude. The user opens a dedicated application and enters information about the budget, family size, and excluded foods. For example, they might enter "Budget 30,000 yen, family of 4, exclude nuts." This input data is locally validated on the terminal to check for any problems with the data format. If there are no problems, the terminal sends this data to the server. The input data is used as input, and validation and transmission are output.

[0448] Step 2:

[0449] The server receives user data sent from the terminal and stores it in the database. The stored data includes the user's budget, family size, and excluded foods. The server analyzes the data and converts it into the necessary data structure. For example, if a user enters "Exclude nuts," nuts are added to the excluded foods list. The input data is user information sent from the terminal, and the output is the information stored in the database.

[0450] Step 3:

[0451] The device collects facial expressions and voice tone when the user inputs information. Equipped with a camera and microphone, the device captures the user's facial expressions and voice data in real time. This data is sent to an emotion engine for analysis. The input is data on facial expressions and voice tone, while the output is analyzed emotion data.

[0452] Step 4:

[0453] The emotion engine analyzes the collected data and sends it to the server. For example, it uses "EmotionRecognition" for facial expression analysis and "VoiceEmotionRecognition" for voice analysis to determine if the user is tired, stressed, etc. This information is then sent to the server. The input data is the collected and analyzed emotion data, and the output is sent to the server.

[0454] Step 5:

[0455] The server generates an optimal menu based on the user's settings and emotional data. For example, if the server detects that the user is tired, it prioritizes suggesting easy-to-prepare dishes. The generated menu data is returned to the terminal and displayed to the user. The input is the user's settings and emotional data, and the output is the generated menu.

[0456] Step 6:

[0457] The server generates a list of necessary ingredients based on the proposed menu. For example, it creates a specific list such as, "For Day 1's chicken curry, you'll need chicken, onions, curry powder, spices, etc." The ingredient list is converted into a detailed data structure and managed. The input is the proposed menu, and the output is the generated ingredient list.

[0458] Step 7:

[0459] The server places an order using the e-commerce site's API based on the confirmed list of ingredients and schedules the delivery date and time. For example, it might set "Purchase chicken, onions, and curry powder on [Month] [Day], and deliver on [Month] [Day]." It calls the API, adds the ingredients to the online shopping cart, and sets the delivery date and time. The input data is the generated list of ingredients, and the output is the completed order information.

[0460] Step 8:

[0461] The server tracks expenses in real time based on purchase history and manages them to stay within budget. It periodically checks budget progress to prevent users from overspending and displays warning messages as needed. For example, it might display, "Current expenses: ¥15,000, remaining budget: ¥15,000." The input data is purchase history, and the output is budget tracking and warning messages.

[0462] Example of a prompt

[0463] User-entered data: Budget of 30,000 yen, family of 4, nuts excluded. Facial expressions and voice tone were analyzed to determine the user was tired. Based on this information, suggest a simple meal plan for the user: "Dinner on day 1: Chicken curry, Day 2: Stir-fried vegetables."

[0464] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0465] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0466] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0467] [Second Embodiment]

[0468] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0469] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0470] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0471] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0472] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0473] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0474] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0475] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0476] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0477] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0478] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0479] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0480] This invention is a system for streamlining household meal management, particularly for dual-income households, automating the process of planning daily menus and purchasing and delivering ingredients within a budget. The system allows users to input their monthly budget, number of people, and foods to be excluded, and then generates daily menus and lists of necessary ingredients. Furthermore, it provides a series of functions for purchasing and delivering ingredients in conjunction with e-commerce sites.

[0481] Program execution steps

[0482] 1. User registration:

[0483] User: Open the dedicated application or web interface and enter your monthly budget, family size, and foods you want to exclude. Specifically, the user might enter information such as "budget of 30,000 yen, family of 4, exclude nuts."

[0484] Terminal: Locally checks the entered data to ensure there are no formatting issues. After the check is complete, the data is sent to the server.

[0485] Server: Receives data sent from terminals, stores it in the database, and manages user-specific settings.

[0486] 2. Menu suggestions:

[0487] Server: Retrieves user configuration information from the database and generates a menu that does not include excluded foods within the budget. For example, it calculates the daily cost for four people with a budget of 30,000 yen and uses an algorithm to select recipes that do not include nuts.

[0488] Terminal: Displays menu suggestions sent from the server to the user. For example, it might display "Dinner on day 1: Chicken curry, Day 2: Stir-fried vegetables."

[0489] 3. Generating the ingredient list:

[0490] Server: Based on the proposed menu, it generates a list of necessary ingredients. For example, it creates a list such as, "For chicken curry, you need chicken, onions, curry powder, spices, etc."

[0491] Terminal: Provides an interface that displays a list of ingredients to the user, allowing the user to review and make necessary corrections.

[0492] 4. Grocery purchase and delivery settings:

[0493] Server: Based on the confirmed list of ingredients, the server connects with the API of the e-commerce site to add ingredients to the cart and schedule the delivery date and time. For example, using the Yahoo! Shopping API, it might set "Purchase chicken, onions, and curry powder on [Month] [Day], and deliver on [Month] [Day]."

[0494] Terminal: Displays a screen to the user to notify them of purchase details and delivery date / time, and for the user to confirm.

[0495] 5. Budget Management:

[0496] Server: Tracks expenses in real time based on purchase history. Checks actual spending against the user's budget and manages it to stay within budget. For example, if spending is likely to exceed the budget near the end of the month, it suggests lower-cost alternative recipes.

[0497] Terminal: Display budget progress and warning messages to the user, allowing them to constantly monitor the budget status.

[0498] Specific example

[0499] For example, a user registers the following conditions:

[0500] Monthly budget: 30,000 yen

[0501] Family size: 4 people

[0502] Excluded items: Nuts

[0503] Based on these conditions, the system performs the following steps: First, at the beginning of the month, it generates a 30-day menu that fits the conditions. For example, a one-week menu might be suggested as follows:

[0504] Day 1: Lunch: Sandwich, Dinner: Chicken Curry

[0505] Day 2: Lunch: Spaghetti, Dinner: Stir-fried vegetables

[0506] ...

[0507] The system then creates a list of ingredients needed for each menu item. For example, it might present a list such as, "For Day 1's chicken curry, you'll need chicken, onions, curry powder, spices, etc." The user reviews this list, makes any necessary adjustments, places an order using the e-commerce site's API, and schedules delivery for the specified day. This entire process allows users to easily decide on their daily menus and automatically purchase the necessary ingredients. The system also manages spending in real time to stay within budget, ensuring a stable diet even amid rising prices.

[0508] The following describes the processing flow.

[0509] Step 1: The user opens the dedicated application or web interface, enters their monthly budget, number of family members, and foods they wish to exclude, and presses the submit button.

[0510] Step 2: The terminal locally validates the entered data and performs error checks. If there are no problems with validation, the data is sent to the server.

[0511] Step 3: The server receives the data sent from the terminal and saves it to the database. For example, it saves information such as "budget of 30,000 yen, family of 4, nuts excluded."

[0512] Step 4: The server retrieves the user's settings information from the database and generates daily menus based on it. It selects menus that do not include excluded foods within the budget and saves them on the server.

[0513] Step 5: The server sends the generated menu data to the terminal and displays it to the user. For example, it might display something like, "Dinner on day 1 is chicken curry, and on day 2 it's stir-fried vegetables."

[0514] Step 6: The user reviews the menu displayed on the device and adds or removes ingredients or adjusts quantities as needed.

[0515] Step 7: The device sends the final ingredient list, which has been reviewed and corrected by the user, to the server.

[0516] Step 8: Based on the confirmed ingredient list, the server adds items to the cart via the e-commerce site's API. Add the necessary ingredients to the cart one by one, and confirm the purchase once all items are available.

[0517] Step 9: After the server completes the purchase process, it schedules the delivery date and time. The schedule takes into account the user's preferred delivery date.

[0518] Step 10: The server sends the delivery details and order contents to the terminal and displays a confirmation screen to the user. The user reviews the delivery date and time and purchase details and makes any necessary changes.

[0519] Step 11: The server tracks expenses in real time based on purchase history. Check the budget and make any necessary adjustments to prevent exceeding the budget.

[0520] Step 12: The device displays budget progress and warning messages to the user. For example, it provides information such as, "Current spending is 15,000 yen, remaining budget is 15,000 yen."

[0521] This allows users to efficiently manage grocery shopping and delivery, and plan their daily meals within their budget.

[0522] (Example 1)

[0523] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0524] Currently, for busy households, such as dual-income families, deciding on daily menus and purchasing and managing ingredients within budget is a significant burden. Furthermore, there is a need for efficient methods of purchasing ingredients while considering budget management, seasonality of ingredients, and nutritional balance. Conventional systems have found it difficult to automate and optimize all of these elements, leaving a considerable amount of effort to the user.

[0525] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0526] In this invention, the server includes means for the user to input a monthly budget, number of people, and foods to be excluded; means for storing the input data; means for generating a menu that does not include excluded foods and stays within the budget based on the stored data; means for generating a list of necessary ingredients based on the generated menu; means for adding products to a cart via an e-commerce site based on the ingredient list and scheduling delivery on a specified date; means for tracking expenses to ensure that monthly expenditures stay within budget based on the stored data; means for checking the format of the input content in real time and transmitting the data if the format is determined to be correct; and means for generating low-cost alternative recipes based on expense tracking. This makes it possible for the user to easily decide on daily menus and automatically purchase and manage ingredients within budget.

[0527] A "user" is someone who uses the system to input monthly budgets, the number of people, and foods to be excluded, and then instructs the system to generate daily menus and lists of necessary ingredients.

[0528] The "means of input" refer to a system that provides an interface for users to input their monthly budget, number of people, and foods they wish to exclude.

[0529] "Means of saving" refers to a system that saves entered data to a storage device such as a database so that it can be used later.

[0530] "Methods for generating menus" refer to algorithms or programs that automatically generate menus within a budget and excluding excluded foods, based on stored data.

[0531] "Means for generating ingredient lists" refers to algorithms or programs that create a list of necessary ingredients based on the generated menu.

[0532] An "e-commerce site" is a platform for selling and buying goods online.

[0533] The "add to cart" method is a system that automates the purchase process of products on e-commerce sites based on a list of ingredients.

[0534] "Means of scheduling delivery" refers to a time management system for delivering products purchased on an e-commerce site to users at a specified date and time.

[0535] A "means of tracking expenses" is a system that monitors and manages expenses in real time based on stored data to ensure that monthly spending stays within budget.

[0536] "A means of checking the format of input content in real time" refers to a system that checks immediately whether there are any problems with the format of the data entered by the user.

[0537] "Means for generating low-cost alternative recipes" are algorithms or programs that, based on cost tracking results, suggest alternative menus that can be prepared at a lower cost if the budget is likely to be exceeded.

[0538] This invention is a system designed to streamline meal management in busy households, such as dual-income families. Specifically, it provides a process for determining daily menus and automating the purchase and delivery of ingredients within a budget.

[0539] This system's program primarily functions through the collaboration of three entities: servers, terminals, and users. The following describes the processing procedures and the hardware and software used for each entity.

[0540] User actions

[0541] Users enter their monthly budget, family size, and foods they wish to exclude through a dedicated application or web interface. For example, a user might enter "Budget: 30,000 yen, Family size: 4, Exclude nuts." The entered data is checked in real time on the terminal and sent to the server only after it is confirmed to be in the correct format.

[0542] Server Processing

[0543] The server performs the following main tasks:

[0544] 1. Data Storage and Management: Receive data submitted by users and store it in a database. For example, use a database system such as MongoDB or MySQL.

[0545] 2. Menu Generation: Based on saved data, menus are generated that stay within budget and do not include excluded foods. Specifically, a generation AI model is used to algorithmically select menus that meet the user's criteria.

[0546] 3. Generating an ingredient list: Based on the suggested menu, generate a list of necessary ingredients. For example, create a list such as, "For chicken curry, you'll need chicken, onions, curry powder, spices, etc."

[0547] 4. Integration with e-commerce sites: Based on the confirmed list of ingredients, the system integrates with the API of an e-commerce site (e.g., Yahoo! Shopping API) to add ingredients to the cart and schedule delivery dates and times.

[0548] 5. Budget Management: Track expenses in real time based on purchase history and check actual spending against the user's budget. If the budget is likely to be exceeded, generate lower-cost alternative recipes.

[0549] Terminal processing

[0550] The terminal (the device operated by the user) provides the following functions:

[0551] 1. Data Verification and Transmission: The data entered by the user is verified in real time, and if there are no formatting issues, it is sent to the server.

[0552] 2. Display of Menus and Ingredient Lists: Menu suggestions and ingredient lists received from the server are displayed in the user interface. For example, it might display "Dinner on Day 1: Chicken Curry, Day 2: Stir-fried Vegetables."

[0553] 3. User Confirmation and Modification: Provide an interface that allows users to review the ingredient list and delivery details and make modifications as needed.

[0554] 4. Displaying budget progress: Display budget progress and warning messages to allow users to always check the budget status.

[0555] Specific example

[0556] For example, a user registers the following conditions:

[0557] Monthly budget: 30,000 yen

[0558] Family size: 4 people

[0559] Excluded items: Nuts

[0560] Based on these conditions, the system performs the following steps: First, it generates a 30-day menu at the beginning of the month within the given conditions. For example, a one-week menu might be suggested as follows:

[0561] Day 1: Lunch: Sandwich, Dinner: Chicken Curry

[0562] Day 2: Lunch: Spaghetti, Dinner: Stir-fried vegetables

[0563] ...

[0564] The system then creates a list of ingredients needed for each menu item. For example, it might present a list such as, "For Day 1's chicken curry, you'll need chicken, onions, curry powder, spices, etc." The user reviews this list, makes any necessary adjustments, places an order using the e-commerce site's API, and schedules delivery for the specified day. This entire process allows users to easily decide on their daily menus and automatically purchase the necessary ingredients. The system also manages spending in real time to stay within budget, ensuring a stable diet even amid rising prices.

[0565] Example of a prompt

[0566] The following are specific examples of prompts to input into a generative AI model:

[0567] "Please create a system to manage meals for a dual-income household with a monthly budget of 30,000 yen for four people. Please provide detailed instructions, including specific menu suggestions when the user specifies nuts as a food to exclude, how to generate a list of necessary ingredients, and the process for purchasing and delivering those ingredients."

[0568] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0569] Program processing flow

[0570] Step 1: User Registration

[0571] input

[0572] User: Enter your budget, family size, and foods you want to exclude using the dedicated application or web interface.

[0573] Specific input example: "Budget 30,000 yen, family of 4, exclude nuts"

[0574] Specific actions

[0575] Terminal: Checks user input in real time. If there are formatting issues, displays an error message and prompts correction. If the format is correct, encrypts the data and sends it to the server.

[0576] output

[0577] The input data is sent to the server.

[0578] Step 2: Save Data

[0579] input

[0580] User data sent from the device

[0581] Specific actions

[0582] Server: Analyzes received data and adds it to the database as a new record. Simultaneously, it generates a user ID and stores it as user initial information. For example, it might use MongoDB or MySQL to manage the data.

[0583] output

[0584] User information is stored in the database.

[0585] Step 3: Menu Generation

[0586] input

[0587] User budget, family size, and exclusion food list retrieved from the database.

[0588] Specific actions

[0589] Server: Executes database queries to retrieve user information. Based on the retrieved data, a generative AI model is used to generate menus that stay within budget and exclude excluded foods. For example, it calculates the daily cost for four people with a monthly budget of 30,000 yen and uses an algorithm to select recipes that do not contain nuts.

[0590] output

[0591] The generated menu information is sent to the terminal.

[0592] Step 4: Menu Display

[0593] input

[0594] Menu information sent from the server

[0595] Specific actions

[0596] Terminal: Displays received menu information on the user interface. For example, it might suggest "Dinner on day 1 is chicken curry" or "Dinner on day 2 is stir-fried vegetables."

[0597] output

[0598] The user checks the menu.

[0599] Step 5: Generating the ingredient list

[0600] input

[0601] Proposed menu information

[0602] Specific actions

[0603] Server: Retrieves the necessary ingredients for each menu item from the database and creates a list. For example, it generates a list such as, "Chicken curry requires chicken, onions, curry powder, spices, etc."

[0604] output

[0605] The generated list of ingredients is sent to the terminal.

[0606] Step 6: Check and correct the ingredient list

[0607] input

[0608] Ingredient list sent from the server

[0609] Specific actions

[0610] Terminal: Displays the ingredient list in the user interface and provides buttons such as "Edit," "Delete," and "Add." When the user makes a modification, the data is sent back to the server.

[0611] output

[0612] Confirmed list of ingredients

[0613] Step 7: Purchase ingredients and set up delivery.

[0614] input

[0615] Confirmed list of ingredients

[0616] Specific actions

[0617] Server: Based on the ingredient list, it connects with the API of an e-commerce site (e.g., Yahoo! Shopping API). For example, it adds "chicken, onion, curry powder" to the cart and sets the delivery date to "Month Day".

[0618] output

[0619] The purchase process is completed on the e-commerce site, and delivery is scheduled.

[0620] Step 8: Notification of purchase details and delivery date / time

[0621] input

[0622] Purchase procedure and delivery schedule information

[0623] Specific actions

[0624] Terminal: The information is displayed as a pop-up message on the user interface, providing a screen for the user to confirm.

[0625] output

[0626] A user verification flag is returned to the server.

[0627] Step 9: Budget management and alternative recipe suggestions

[0628] input

[0629] Purchase history data

[0630] Specific actions

[0631] Server: Tracks expenses in real time and manages spending against the user's budget. If the budget is likely to be exceeded, it uses a generative AI model to suggest lower-cost alternative recipes.

[0632] output

[0633] An alternative recipe is sent to the device.

[0634] Step 10: Displaying budget progress

[0635] input

[0636] Budget progress and warning messages sent from the server

[0637] Specific actions

[0638] Terminal: The budget management screen displays information such as "Current expenses: 28,000 yen" and "Remaining budget: 2,000 yen," and a warning icon flashes to notify the user.

[0639] output

[0640] User verification and budget adjustment

[0641] (Application Example 1)

[0642] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0643] There are challenges in streamlining household meal management, such as automating daily meal planning, grocery purchasing, and delivery within budget for dual-income households. Furthermore, it's necessary to minimize the effort users have to manually check and adjust ingredients, and to provide real-time updates on budget progress and warning messages.

[0644] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0645] In this invention, the server includes means for the user to input a monthly budget, number of people, and foods to be excluded; means for storing the input data; means for generating a menu that does not include excluded foods and stays within the budget based on the stored data; means for generating a list of necessary ingredients based on the generated menu; means for adding products to an online shopping cart based on the list of necessary ingredients and scheduling delivery on a specified date; means for tracking expenses based on the stored data to ensure that monthly expenditures stay within the budget; and means for providing an interface that allows the user to check the budget progress and warning messages. This automates the daily tasks of deciding on menus, purchasing ingredients, and delivery, and enables more efficient budget management.

[0646] "A means for users to input monthly budgets, number of people, and foods to exclude" refers to an interface for inputting basic data for monthly meal planning, taking into account constraints such as family structure, budget, and allergies.

[0647] "Means of saving entered data" refers to a function that stores information entered by the user on a cloud server or database, making it accessible later.

[0648] "A means of generating a menu that stays within budget and excludes excluded foods based on saved data" refers to an algorithm that uses saved user data to automatically plan a menu that stays within budget and excludes specified foods.

[0649] "A means of generating a list of necessary ingredients based on a generated menu" refers to a function that creates a list of ingredients needed to prepare a meal according to a planned menu.

[0650] "A method for adding products to an online shopping cart based on a list of necessary ingredients and scheduling delivery on a specified date" refers to a function that uses the API of an online shopping site to add products to the cart based on the created ingredient list and arranges delivery at the date and time desired by the user.

[0651] "A means of tracking expenses to ensure monthly spending stays within budget based on saved data" refers to a feature that monitors purchase history and current spending in real time and adjusts spending to stay within budget, in order to help users manage their budget.

[0652] "Means of providing an interface that allows users to check budget progress and warning messages" refers to a function that provides a screen through an application or web browser that allows users to check budget progress and related warning messages in real time.

[0653] The "means for generating menus that also consider the seasonality and nutritional balance of ingredients, and further call external e-commerce site APIs for purchasing products" refers to an algorithm and function that generates menus that include seasonal ingredients and nutritional balance, and then executes the process of purchasing ingredients using APIs from external shopping sites.

[0654] "Means of providing an interface for users to view, add, delete, or adjust the quantity of ingredients in an ingredient list" refers to a function that provides an interactive screen for users to add or delete items and adjust quantities as needed, based on the ingredient list they have created.

[0655] "A means of inputting prompt text into a generative AI model to generate recipes and ingredient information" refers to a function that executes a process in which the user inputs specific conditions or preferences in text format, and the generative AI model generates the optimal recipe and related ingredient information.

[0656] The system of this invention provides a comprehensive approach to streamline family meal management. Specifically, it supports an interface in which the user inputs a monthly budget, the number of family members, and foods to be excluded, and stores and uses this information. Data entered via a device (such as a smartphone or PC) is stored in a cloud-based database.

[0657] The server generates menus that fit within the budget and do not include excluded foods, based on the stored data. For example, if the registered data is "monthly budget of 30,000 yen, family size of 4, excluded foods: nuts," the server will generate and recommend appropriate recipes that meet those conditions. Based on the generated menus, the server automatically generates a list of necessary ingredients. This ingredient list provides an interface that allows the user to review, add, delete, and adjust the quantities of ingredients.

[0658] Furthermore, based on the list of necessary ingredients, the server interacts with the API of an online shopping site (for example, a hypothetical e-commerce API), adds the items to the cart, and schedules delivery of the ingredients at a date and time specified by the user. This allows users to obtain the necessary ingredients without leaving their homes.

[0659] Based on the stored data, the server tracks monthly spending in real time and manages it to stay within budget. It also provides an interface where users can check budget progress and receive warning messages, ensuring that spending is always properly managed. For example, it will issue an alert if there is a possibility of exceeding the budget near the end of the month.

[0660] In addition, the server creates menus that take seasonality and nutritional balance into consideration, supporting users' health management. The generated list of ingredients automates the process of purchasing by calling APIs of external e-commerce sites. Users can easily perform these operations through the interface.

[0661] As a concrete example, here is an example of a prompt message that the user might enter:

[0662] "For the development of a food and beverage management application, please propose a 30-day menu based on the following input information: budget of 30,000 yen, family size of 4, and exclude nuts. Additionally, please generate a list of ingredients required for each menu."

[0663] The main hardware / software used in implementing this invention includes a cloud-based database, server-side programming languages ​​(e.g., Python and Flask), and libraries to facilitate communication with external APIs (e.g., Requests). This reduces user effort and enables an efficient meal management system.

[0664] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0665] Step 1:

[0666] Users enter their budget, family size, and foods they wish to exclude using an application or web interface. Once the data is entered, it is sent from the device to the server. The entered data includes monthly budget, family composition, and allergy information.

[0667] Input: Budget, number of family members, foods to exclude

[0668] Output: User data sent to the server

[0669] Step 2:

[0670] The server saves the received data to a database. Once stored in the database, the user's settings become available for subsequent processing.

[0671] Input: User data sent to the server

[0672] Output: User information stored in the database

[0673] Step 3:

[0674] The server generates a menu that fits within the budget and excludes excluded foods, based on stored user information. This process also applies an algorithm that takes into account the seasonality and nutritional balance of ingredients.

[0675] Input: User information stored in the database

[0676] Output: Generated menu

[0677] Step 4:

[0678] The server creates a list of necessary ingredients based on the generated menu. This list details the ingredients required for each menu recipe, including quantities and units.

[0679] Input: Generated menu

[0680] Output: List of required ingredients

[0681] Step 5:

[0682] The ingredient list created by the server is sent to the user's terminal, where the user can review it. The user can add, delete, or adjust the contents of the ingredient list through the interface. The user's actions are resent to the server.

[0683] Input: List of required ingredients

[0684] Output: User-reviewed and adjusted ingredient list

[0685] Step 6:

[0686] Based on the final list of ingredients, the server calls the e-commerce site's API to add the ingredients to the cart and schedule delivery for the date and time specified by the user. The results of the API calls are stored on the server and notified to the user.

[0687] Input: User-confirmed and adjusted ingredient list

[0688] Output: Purchase information and delivery schedule in e-commerce

[0689] Step 7:

[0690] The server tracks purchase history and current spending in real time, managing it to stay within budget. If the budget is likely to be exceeded, it notifies the user with alternatives or warnings.

[0691] Input: Purchase history, current spending data

[0692] Output: Spending tracking information, warning messages

[0693] Step 8:

[0694] Users can check budget progress and warning messages through the interface. The server provides this information in real time, allowing users to always be aware of their current budget status.

[0695] Input: Expense tracking information, warning messages

[0696] Output: Budget progress and warning messages displayed in the user interface.

[0697] An example of a specific prompt message is as follows:

[0698] "For the development of a food and beverage management application, please propose a 30-day menu based on the following input information: budget of 30,000 yen, family size of 4, and exclude nuts. Additionally, please generate a list of ingredients required for each menu."

[0699] The above is a description of the specific processing steps.

[0700] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0701] This invention is a system designed to streamline household meal management, particularly for dual-income households, automating the process of planning daily menus and purchasing and delivering ingredients within a budget. Furthermore, by incorporating an emotion engine that recognizes user emotions, this system provides a more personalized service by offering menu suggestions that take the user's emotional state into account.

[0702] Program execution steps

[0703] 1. User registration:

[0704] User: Open the dedicated application or web interface, enter your monthly budget, family size, and foods you want to exclude, and press the submit button. For example, enter "Budget 30,000 yen, family of 4, exclude nuts."

[0705] Terminal: Validates the entered data locally and performs error checks. If there are no problems, the data is sent to the server.

[0706] Server: Receives data sent from terminals and stores it in the database. Manages data based on user configuration information.

[0707] 2. Emotion recognition:

[0708] Terminal: When a user enters data, it collects facial expressions, voice tone, and text content.

[0709] Emotion Engine: Analyzes collected data to recognize the user's emotions. For example, it uses facial recognition and voice analysis technologies to extract emotions such as fatigue or stress.

[0710] Server: Receives the analysis results from the emotion engine and saves them to the database.

[0711] 3. Menu suggestions:

[0712] Server: Based on user configuration information and sentiment data, the server generates an optimal menu considering the user's emotional state. For example, if the server determines that the user is tired, it will suggest easy-to-prepare dishes.

[0713] Terminal: Displays menu suggestions sent from the server to the user. For example, it might suggest "Dinner on day 1: Chicken curry, Day 2: Stir-fried vegetables."

[0714] 4. Generating the ingredient list:

[0715] Server: Based on the proposed menu, it generates a list of necessary ingredients. For example, it creates a list such as, "For chicken curry, you need chicken, onions, curry powder, spices, etc."

[0716] Terminal: Provides an interface that displays a list of ingredients to the user, allowing the user to review and make necessary corrections.

[0717] 5. Grocery purchase and delivery settings:

[0718] Server: Based on the confirmed list of ingredients, the server adds items to the cart via the e-commerce site's API and schedules the delivery date and time. For example, it might set "Purchase chicken, onions, and curry powder on [Month] [Day], and deliver on [Month] [Day]."

[0719] Terminal: Displays a screen to the user to notify them of purchase details and delivery date / time, and for the user to confirm.

[0720] 6. Budget Management:

[0721] Server: Tracks expenses in real time based on purchase history. Checks actual spending against the user's budget and manages spending to stay within budget.

[0722] Terminal: Displays budget progress and warning messages to the user. For example, it might display "Current expenses are 15,000 yen, remaining budget is 15,000 yen."

[0723] Specific example

[0724] For example, a user registers the following conditions:

[0725] Monthly budget: 30,000 yen

[0726] Family size: 4 people

[0727] Excluded items: Nuts

[0728] Based on these conditions, when the user enters information today, the device collects the user's facial expressions and voice tone, and the emotion engine analyzes them. The emotion engine recognizes that the user is tired. Based on this information, the server suggests a menu that can be easily prepared: "Dinner on day 1: Chicken curry, Day 2: Stir-fried vegetables."

[0729] The system then creates a list of ingredients needed for each menu item, presenting a list such as, "For Day 1's chicken curry, you'll need chicken, onions, curry powder, spices, etc." The user reviews this list, makes any necessary adjustments, places an order using the e-commerce site's API, and schedules delivery for the specified date.

[0730] Furthermore, the system manages spending in real time to keep it within budget, displaying information such as "Current spending: ¥15,000, remaining budget: ¥15,000," allowing users to constantly monitor their budget status. This entire process enables users to effortlessly decide on daily menus and automatically purchase necessary ingredients. In addition, by combining this with an emotion engine, the system can provide optimal suggestions tailored to the user's emotional state.

[0731] The following describes the processing flow.

[0732] Step 1: The user opens the dedicated application or web interface, enters their monthly budget, number of family members, and foods they want to exclude, and presses the submit button. For example, they might enter "Budget 30,000 yen, family of 4, exclude nuts."

[0733] Step 2: The terminal locally validates the entered data and performs error checks. If there are no errors, it sends the input data to the server.

[0734] Step 3: The server receives the data sent from the terminal and saves it to the database. This saves the user's configuration information for management purposes.

[0735] Step 4: The device collects the user's facial expressions, voice tone, and entered text. This data provides the foundational information needed to understand the user's emotional state.

[0736] Step 5: The emotion engine analyzes the collected emotion data to recognize the user's emotional state. For example, it uses facial recognition and voice analysis technologies to determine whether the user is tired, stressed, etc.

[0737] Step 6: The server receives the analysis results from the emotion engine and stores them in the database. This allows for tracking and long-term management of the user's emotional state.

[0738] Step 7: The server generates a menu based on the user's settings and emotional data. It also takes the user's emotional state into consideration; for example, if the user is tired, it selects dishes that are easy to prepare.

[0739] Step 8: The server sends the generated menu data to the terminal and displays it to the user. Here, it visually presents specific details such as "Dinner on day 1 is chicken curry, and on day 2 is stir-fried vegetables."

[0740] Step 9: The user reviews the menu displayed on their device and adds or removes ingredients or adjusts quantities as needed. After reviewing, they send the changes to the server.

[0741] Step 10: Based on the final list of ingredients received from the user, the server uses the e-commerce site's API to add items to the cart. Each ingredient is added to the cart sequentially, and the purchase process is completed.

[0742] Step 11: After the server completes the purchase process, it schedules the delivery date and time based on the user's preferences. For example, it might set "Purchase chicken, onions, and curry powder on [Month] [Day], and deliver on [Month] [Day]."

[0743] Step 12: The server sends the purchase details and delivery date and time to the terminal and displays a confirmation screen to the user. The user reviews the order details and delivery schedule and makes any necessary changes.

[0744] Step 13: The server tracks costs in real time based on purchase history. It manages to prevent budget overruns and suggests alternative recipes if shortages are likely.

[0745] Step 14: The device displays budget progress and warning messages to the user. For example, it provides information such as "Current expenses are 15,000 yen, remaining budget is 15,000 yen" to ensure the user is always aware of their budget status.

[0746] This allows users to easily decide on their daily menus and efficiently purchase and have necessary ingredients delivered. Furthermore, by utilizing an emotion engine, the system can provide optimal menu suggestions tailored to the user's emotional state.

[0747] (Example 2)

[0748] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0749] This invention aims to reduce the burden on users, particularly in dual-income households, by automating the processes of daily meal planning, purchasing and delivery of ingredients within budget, and providing personalized services that take into account the user's emotional state. Furthermore, it aims to provide a system that integrates these processes, including meal suggestions that consider the season and nutritional balance, and budget management.

[0750] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0751] In this invention, the server includes means for storing user input data, means for analyzing emotional data, and means for generating menus and creating ingredient lists. This enables menu suggestions that take into account the user's emotional state, as well as automated ingredient purchasing and delivery.

[0752] "User input" refers to the process where users enter necessary information, such as monthly budgets, number of people, and foods they wish to exclude, through a dedicated application or web interface.

[0753] "Emotion recognition" is the process of recognizing a user's emotional state by analyzing their facial expressions, tone of voice, text content, and other factors.

[0754] "Menu generation" refers to the automatic creation of a menu that fits within the budget and does not exclude certain foods, based on the user's settings and sentiment data.

[0755] "Ingredient list generation" refers to creating a list of necessary ingredients based on the proposed menu.

[0756] An "e-commerce site" is an online platform for purchasing goods via the internet.

[0757] "Tracking" refers to the real-time management of expenses to ensure that a user's monthly spending stays within budget.

[0758] An "interface" is the user interface or operating environment through which a user interacts with a system, performing tasks such as inputting data, confirming information, and making corrections.

[0759] This invention is a system designed to streamline household meal management, particularly for dual-income households, automating the process of planning daily menus and purchasing and delivering ingredients within a budget. Furthermore, by incorporating an emotion engine that recognizes user emotions, this system provides a more personalized service by offering menu suggestions that take the user's emotional state into account.

[0760] User input and data storage

[0761] First, the user uses a dedicated application or web interface to enter their monthly budget, family size, and foods they wish to exclude. This data is validated and error-checked on the device before being sent to the server and stored in the database. For example, information such as "budget 30,000 yen, family of 4, exclude nuts" might be entered.

[0762] emotion recognition

[0763] When a user enters information, the device collects the user's facial expressions, voice tone, and text content. This collected data is analyzed by an emotion engine. Specifically, it uses facial recognition technology (e.g., Google Cloud Vision) and voice analysis technology (e.g., IBM Watson) to recognize the user's emotions. The analysis results are sent to a server and stored in a database.

[0764] Menu generation

[0765] The server generates an optimal menu based on saved user settings and emotional data. Considering the user's emotional state, for example, if the user is tired, a menu that is easy to prepare will be suggested. The suggested menu is sent to the terminal and displayed to the user. Specifically, a menu such as "Dinner on day 1: Chicken curry, Day 2: Stir-fried vegetables" might be suggested.

[0766] Generating and purchasing a list of ingredients.

[0767] The server generates a list of necessary ingredients based on the suggested menu. This list is displayed on the terminal for user confirmation and modification. The server then adds the items to the cart via the e-commerce site's API and schedules the delivery date and time. For example, it might be set to "Purchase chicken, onions, and curry powder on [Month] [Day], and deliver on [Month] [Day]."

[0768] Budget management and expense tracking

[0769] The server tracks purchase history in real time and checks actual spending against the user's budget. For example, it might display "Current spending: 15,000 yen, remaining budget: 15,000 yen." This allows users to constantly monitor their budget progress.

[0770] Specific example

[0771] For example, a user registers the following conditions:

[0772] Monthly budget: 30,000 yen

[0773] Family size: 4 people

[0774] Excluded items: Nuts

[0775] When you enter your information today, the device collects your facial expressions and voice tone, and an emotion engine analyzes this data. The emotion engine recognizes that the user is tired, and based on this information, the server suggests a menu that is easy to prepare. For example, it might suggest "Chicken curry for dinner on day 1, and stir-fried vegetables on day 2." A list of ingredients is also generated, for example, "For chicken curry on day 1, you'll need chicken, onions, curry powder, spices, etc." An order is then placed through an e-commerce site, and delivery is scheduled for the specified date. Finally, purchase history is tracked in real time, ensuring that your budget is properly managed.

[0776] Example of a prompt

[0777] "If a user has a monthly budget of 30,000 yen, a family of four, and wants to exclude nuts, and they are perceived as tired when deciding what to eat today, suggest chicken curry or stir-fried vegetables as the best menu for them."

[0778] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0779] Program execution steps

[0780] Step 1: User Registration

[0781] The user opens a dedicated application or web interface and enters their monthly budget, the number of family members, and the foods they want to exclude.

[0782] The terminal locally validates the input data and performs error checks. For example, it verifies that the budget and number of family members are in numerical format and that no invalid characters are included. If there are no problems, it sends the data to the server.

[0783] Input: User enters information such as "budget of 30,000 yen, family of 4, excludes nuts".

[0784] Processing: Local validation and error checking.

[0785] Output: The validated input data is sent to the server.

[0786] The server saves the received data to the database and returns a registration completion message to the user. If the entire process is successful, feedback such as "Registration Complete" is sent to the user.

[0787] Input: Validated data from the terminal.

[0788] Processing: Save data to the database.

[0789] Output: Send a registration completion message to the device.

[0790] Step 2: Emotion Recognition

[0791] The device collects facial expressions, voice tone, and text content when the user enters data.

[0792] Input: User's facial expression, voice tone, and input text.

[0793] Processing: Emotional data is collected by integrating facial expressions, voice tone, and text.

[0794] Output: Collected data is sent to the emotion engine.

[0795] The emotion engine analyzes collected data to recognize the user's emotions. Specifically, it uses facial recognition and voice analysis technologies to extract emotions such as fatigue and stress.

[0796] Input: Facial expressions, voice tone, and text data from the device.

[0797] Processing: Sentiment analysis using machine learning models.

[0798] Output: Send emotion recognition results to the server.

[0799] The server receives the analysis results from the emotion engine and stores them in the database.

[0800] Input: Analysis results from the emotion engine.

[0801] Processing: Save to database.

[0802] Output: Feedback of saved results.

[0803] Step 3: Menu Suggestions

[0804] The server generates an optimal menu based on the user's settings and emotional data. For example, if the server determines that the user is tired, it will suggest easy-to-prepare dishes.

[0805] Input: Configuration information and sentiment data.

[0806] Processing: Data calculations to generate menus by integrating configuration information and sentiment data.

[0807] Output: Sends the generated menu data to the terminal.

[0808] The terminal displays menu suggestions sent from the server to the user.

[0809] Input: Menu data from the server.

[0810] Processing: Reflecting menu data in the interface.

[0811] Output: Displays menus to the user, such as "Dinner on day 1 is chicken curry, and on day 2 is stir-fried vegetables."

[0812] Step 4: Generating the ingredient list

[0813] The server generates a list of necessary ingredients based on the proposed menu. For example, it might list "Chicken curry requires chicken, onions, curry powder, spices, etc."

[0814] Input: Suggested menu data.

[0815] Processing: Based on the menu data, retrieve a list of necessary ingredients from the database and create a list.

[0816] Output: Sends the generated ingredient list data to the terminal.

[0817] The terminal displays a list of ingredients to the user and provides an interface for the user to review and make necessary corrections.

[0818] Input: Ingredient list data from the server.

[0819] Processing: Reflecting ingredient list data in the interface.

[0820] Output: Displays a list of ingredients to the user, allowing for confirmation and modification.

[0821] Step 5: Purchase ingredients and set up delivery.

[0822] Based on the confirmed list of ingredients, the server adds items to the cart via the e-commerce site's API and schedules the delivery date and time.

[0823] Input: User-modified list of confirmed ingredients.

[0824] Processing: Add products to the cart and create a delivery schedule via the e-commerce site's API.

[0825] Output: Sends purchase and delivery details to the device.

[0826] The device notifies the user of the purchase details and delivery date and time, and displays a screen for the user to confirm.

[0827] Input: Purchase and delivery details data from the server.

[0828] Processing: Reflects purchase details and delivery schedule in the interface.

[0829] Output: Notify the user and display a confirmation screen.

[0830] Step 6: Budget Management

[0831] The server tracks expenses in real time based on purchase history. It checks actual spending against the user's budget and manages it to stay within budget.

[0832] Input: Purchase history data.

[0833] Processing: Real-time cost tracking and budget checking.

[0834] Output: Sends budget progress and warning messages to the terminal.

[0835] The terminal displays budget progress and warning messages to the user.

[0836] Input: Budget progress and warning messages from the server.

[0837] Processing: Reflect budget progress and warning messages in the interface.

[0838] Output: Displays budget progress and warning messages to the user.

[0839] (Application Example 2)

[0840] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0841] For dual-income households and busy modern people, deciding on daily menus is a significant burden, and purchasing ingredients within budget and cooking efficiently is a difficult challenge. Furthermore, failing to consider the user's emotional state during these tasks can increase stress and burden. In particular, the process of creating menus and purchasing ingredients while considering factors such as family size, food preferences, and allergy information is complex, and there is a need for an appropriate system to streamline these processes.

[0842] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for the user to input a monthly budget, number of people, and foods to be excluded; means for storing the input data; means for generating a menu that does not include excluded foods within the budget based on the stored data; means for generating a list of necessary ingredients based on the generated menu; means for adding products to a cart via an e-commerce site based on the ingredient list and scheduling delivery on a specified date; means for tracking expenses so that monthly expenditures stay within the budget based on the stored data; means for recognizing the user's emotions using facial expressions and voice tone; and means for suggesting a personalized menu based on the user's emotional data. This enables the suggestion of optimal menus with less stress according to the user's emotional state, as well as efficient purchase and management of ingredients.

[0843] A "user" is an individual or family member who uses the system.

[0844] A "budget" is the amount of money that is allowed to be spent on food and meals in a month.

[0845] "Number of people" refers to the number of family members or residents in a household who will be using the system for meal management.

[0846] "Foods to be excluded" are ingredients that users wish to avoid consuming for reasons such as health conditions, allergies, or personal preferences.

[0847] An "input method" is an interface that allows a user to provide the system with the information they need.

[0848] A "storage system" is a system that retains entered data and manages it in a way that allows it to be referenced when needed.

[0849] A "menu" is a plan that outlines the types and contents of meals a user will eat.

[0850] A "menu generation method" is a means of automatically creating a menu that does not exclude any foods, within the user's budget, based on the input data.

[0851] A "food ingredient list" is a list of ingredients needed based on the generated menu.

[0852] An "e-commerce site" is an online platform for purchasing goods via the internet.

[0853] A "delivery scheduling method" is a means of ensuring that food ingredients are delivered at the date and time specified by the user.

[0854] "Expense tracking methods" are means of monitoring and managing expenses to ensure that monthly expenditures stay within budget.

[0855] "Emotion recognition means" refers to technology that identifies emotions from a user's facial expressions and voice tone.

[0856] A "personalized menu" is an individually optimized meal plan that takes into account the user's emotional data.

[0857] This invention is a system that streamlines household meal management, automatically determining daily menus and purchasing and delivering ingredients within budget, especially for dual-income households. Furthermore, by incorporating an emotion engine that recognizes user emotions, it provides menu suggestions that take into account the user's emotional state, offering a more personalized service.

[0858] Hardware and software to be used

[0859] Smartphone: A device on which applications are installed.

[0860] Emotion recognition engine: "EmotionRecognition" analyzes facial expressions and "VoiceEmotionRecognition" analyzes voice tone.

[0861] Server: A server that maintains the database and provides APIs.

[0862] E-commerce API: An external API for purchasing groceries online.

[0863] Explanation of the process

[0864] 1. User registration:

[0865] Users input basic information such as their monthly budget, family size, and foods they wish to exclude using a dedicated application. This enables personalized meal planning and ingredient management for each household.

[0866] 2. Emotion recognition:

[0867] When a user enters information, the device collects facial expressions and voice tone, which are then analyzed by an emotion recognition engine. For example, facial recognition and voice analysis technologies can be used to extract emotional states such as whether the user is tired or stressed.

[0868] 3. Menu suggestions:

[0869] The server generates an optimal menu based on the user's settings and emotional data. For example, if it determines that the user is tired, it can suggest easy-to-prepare dishes.

[0870] 4. Generating the ingredient list:

[0871] Based on the suggested menu, a list of necessary ingredients is generated. This allows the user to efficiently prepare the required ingredients.

[0872] 5. Grocery purchase and delivery settings:

[0873] Based on the confirmed list of ingredients, the server adds items to the cart via the API of online e-commerce sites and schedules delivery dates and times. This allows users to obtain ingredients without any hassle.

[0874] 6. Budget Management:

[0875] The server tracks expenses based on purchase history to ensure they stay within budget and displays the user's current spending status. For example, it might show information such as "Current spending: ¥15,000, remaining budget: ¥15,000" in real time.

[0876] Specific example

[0877] For example, a user registers the following conditions:

[0878] Monthly budget: 30,000 yen

[0879] Family size: 4 people

[0880] Excluded items: Nuts

[0881] When a user enters information today, the device collects the user's facial expressions and voice tone, which are then analyzed by an emotion engine. The emotion engine recognizes that the user is tired. Based on this information, the server suggests easy-to-prepare meals, such as "Dinner on Day 1: Chicken Curry, Day 2: Stir-fried Vegetables." The system then creates a list of ingredients needed for each meal, presenting a list such as, "For Day 1's Chicken Curry: Chicken, onions, curry powder, spices..." The user reviews this list, makes any necessary adjustments, places an order using the e-commerce site's API, and schedules delivery for the specified day. Furthermore, the system manages spending in real time to keep it within budget, displaying information such as, "Current spending: 15,000 yen, remaining budget: 15,000 yen," allowing the user to constantly monitor their budget status.

[0882] Example of a prompt

[0883] User-entered data: Budget of 30,000 yen, family of 4, nuts excluded. Facial expressions and voice tone were analyzed to determine the user was tired. Based on this information, suggest a simple meal plan for the user: "Dinner on day 1: Chicken curry, Day 2: Stir-fried vegetables."

[0884] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0885] Step 1:

[0886] The user enters their monthly budget, number of people, and foods they want to exclude. The user opens a dedicated application and enters information about the budget, family size, and excluded foods. For example, they might enter "Budget 30,000 yen, family of 4, exclude nuts." This input data is locally validated on the terminal to check for any problems with the data format. If there are no problems, the terminal sends this data to the server. The input data is used as input, and validation and transmission are output.

[0887] Step 2:

[0888] The server receives user data sent from the terminal and stores it in the database. The stored data includes the user's budget, family size, and excluded foods. The server analyzes the data and converts it into the necessary data structure. For example, if a user enters "Exclude nuts," nuts are added to the excluded foods list. The input data is user information sent from the terminal, and the output is the information stored in the database.

[0889] Step 3:

[0890] The device collects facial expressions and voice tone when the user inputs information. Equipped with a camera and microphone, the device captures the user's facial expressions and voice data in real time. This data is sent to an emotion engine for analysis. The input is data on facial expressions and voice tone, while the output is analyzed emotion data.

[0891] Step 4:

[0892] The emotion engine analyzes the collected data and sends it to the server. For example, it uses "EmotionRecognition" for facial expression analysis and "VoiceEmotionRecognition" for voice analysis to determine if the user is tired, stressed, etc. This information is then sent to the server. The input data is the collected and analyzed emotion data, and the output is sent to the server.

[0893] Step 5:

[0894] The server generates an optimal menu based on the user's settings and emotional data. For example, if the server detects that the user is tired, it prioritizes suggesting easy-to-prepare dishes. The generated menu data is returned to the terminal and displayed to the user. The input is the user's settings and emotional data, and the output is the generated menu.

[0895] Step 6:

[0896] The server generates a list of necessary ingredients based on the proposed menu. For example, it creates a specific list such as, "For Day 1's chicken curry, you'll need chicken, onions, curry powder, spices, etc." The ingredient list is converted into a detailed data structure and managed. The input is the proposed menu, and the output is the generated ingredient list.

[0897] Step 7:

[0898] The server places an order using the e-commerce site's API based on the confirmed list of ingredients and schedules the delivery date and time. For example, it might set "Purchase chicken, onions, and curry powder on [Month] [Day], and deliver on [Month] [Day]." It calls the API, adds the ingredients to the online shopping cart, and sets the delivery date and time. The input data is the generated list of ingredients, and the output is the completed order information.

[0899] Step 8:

[0900] The server tracks expenses in real time based on purchase history and manages them to stay within budget. It periodically checks budget progress to prevent users from overspending and displays warning messages as needed. For example, it might display, "Current expenses: ¥15,000, remaining budget: ¥15,000." The input data is purchase history, and the output is budget tracking and warning messages.

[0901] Example of a prompt

[0902] User-entered data: Budget of 30,000 yen, family of 4, nuts excluded. Facial expressions and voice tone were analyzed to determine the user was tired. Based on this information, suggest a simple meal plan for the user: "Dinner on day 1: Chicken curry, Day 2: Stir-fried vegetables."

[0903] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0904] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0905] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0906] [Third Embodiment]

[0907] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0908] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0909] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0910] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0911] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0912] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0913] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0914] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0915] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0916] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0917] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0918] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0919] This invention is a system for streamlining household meal management, particularly for dual-income households, automating the process of planning daily menus and purchasing and delivering ingredients within a budget. The system allows users to input their monthly budget, number of people, and foods to be excluded, and then generates daily menus and lists of necessary ingredients. Furthermore, it provides a series of functions for purchasing and delivering ingredients in conjunction with e-commerce sites.

[0920] Program execution steps

[0921] 1. User registration:

[0922] User: Open the dedicated application or web interface and enter your monthly budget, family size, and foods you want to exclude. Specifically, the user might enter information such as "budget of 30,000 yen, family of 4, exclude nuts."

[0923] Terminal: Locally checks the entered data to ensure there are no formatting issues. After the check is complete, the data is sent to the server.

[0924] Server: Receives data sent from terminals, stores it in the database, and manages user-specific settings.

[0925] 2. Menu suggestions:

[0926] Server: Retrieves user configuration information from the database and generates a menu that does not include excluded foods within the budget. For example, it calculates the daily cost for four people with a budget of 30,000 yen and uses an algorithm to select recipes that do not include nuts.

[0927] Terminal: Displays menu suggestions sent from the server to the user. For example, it might display "Dinner on day 1: Chicken curry, Day 2: Stir-fried vegetables."

[0928] 3. Generating the ingredient list:

[0929] Server: Based on the proposed menu, it generates a list of necessary ingredients. For example, it creates a list such as, "For chicken curry, you need chicken, onions, curry powder, spices, etc."

[0930] Terminal: Provides an interface that displays a list of ingredients to the user, allowing the user to review and make necessary corrections.

[0931] 4. Grocery purchase and delivery settings:

[0932] Server: Based on the confirmed list of ingredients, the server connects with the API of the e-commerce site to add ingredients to the cart and schedule the delivery date and time. For example, using the Yahoo! Shopping API, it might set "Purchase chicken, onions, and curry powder on [Month] [Day], and deliver on [Month] [Day]."

[0933] Terminal: Displays a screen to the user to notify them of purchase details and delivery date / time, and for the user to confirm.

[0934] 5. Budget Management:

[0935] Server: Tracks expenses in real time based on purchase history. Checks actual spending against the user's budget and manages it to stay within budget. For example, if spending is likely to exceed the budget near the end of the month, it suggests lower-cost alternative recipes.

[0936] Terminal: Display budget progress and warning messages to the user, allowing them to constantly monitor the budget status.

[0937] Specific example

[0938] For example, a user registers the following conditions:

[0939] Monthly budget: 30,000 yen

[0940] Family size: 4 people

[0941] Excluded items: Nuts

[0942] Based on these conditions, the system performs the following steps: First, at the beginning of the month, it generates a 30-day menu that fits the conditions. For example, a one-week menu might be suggested as follows:

[0943] Day 1: Lunch: Sandwich, Dinner: Chicken Curry

[0944] Day 2: Lunch: Spaghetti, Dinner: Stir-fried vegetables

[0945] ...

[0946] The system then creates a list of ingredients needed for each menu item. For example, it might present a list such as, "For Day 1's chicken curry, you'll need chicken, onions, curry powder, spices, etc." The user reviews this list, makes any necessary adjustments, places an order using the e-commerce site's API, and schedules delivery for the specified day. This entire process allows users to easily decide on their daily menus and automatically purchase the necessary ingredients. The system also manages spending in real time to stay within budget, ensuring a stable diet even amid rising prices.

[0947] The following describes the processing flow.

[0948] Step 1: The user opens the dedicated application or web interface, enters their monthly budget, number of family members, and foods they wish to exclude, and presses the submit button.

[0949] Step 2: The terminal locally validates the entered data and performs error checks. If there are no problems with validation, the data is sent to the server.

[0950] Step 3: The server receives the data sent from the terminal and saves it to the database. For example, it saves information such as "budget of 30,000 yen, family of 4, nuts excluded."

[0951] Step 4: The server retrieves the user's settings information from the database and generates daily menus based on it. It selects menus that do not include excluded foods within the budget and saves them on the server.

[0952] Step 5: The server sends the generated menu data to the terminal and displays it to the user. For example, it might display something like, "Dinner on day 1 is chicken curry, and on day 2 it's stir-fried vegetables."

[0953] Step 6: The user reviews the menu displayed on the device and adds or removes ingredients or adjusts quantities as needed.

[0954] Step 7: The device sends the final ingredient list, which has been reviewed and corrected by the user, to the server.

[0955] Step 8: Based on the confirmed ingredient list, the server adds items to the cart via the e-commerce site's API. Add the necessary ingredients to the cart one by one, and confirm the purchase once all items are available.

[0956] Step 9: After the server completes the purchase process, it schedules the delivery date and time. The schedule takes into account the user's preferred delivery date.

[0957] Step 10: The server sends the delivery details and order contents to the terminal and displays a confirmation screen to the user. The user reviews the delivery date and time and purchase details and makes any necessary changes.

[0958] Step 11: The server tracks expenses in real time based on purchase history. Check the budget and make any necessary adjustments to prevent exceeding the budget.

[0959] Step 12: The device displays budget progress and warning messages to the user. For example, it provides information such as, "Current spending is 15,000 yen, remaining budget is 15,000 yen."

[0960] This allows users to efficiently manage grocery shopping and delivery, and plan their daily meals within their budget.

[0961] (Example 1)

[0962] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0963] Currently, for busy households, such as dual-income families, deciding on daily menus and purchasing and managing ingredients within budget is a significant burden. Furthermore, there is a need for efficient methods of purchasing ingredients while considering budget management, seasonality of ingredients, and nutritional balance. Conventional systems have found it difficult to automate and optimize all of these elements, leaving a considerable amount of effort to the user.

[0964] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0965] In this invention, the server includes means for the user to input a monthly budget, number of people, and foods to be excluded; means for storing the input data; means for generating a menu that does not include excluded foods and stays within the budget based on the stored data; means for generating a list of necessary ingredients based on the generated menu; means for adding products to a cart via an e-commerce site based on the ingredient list and scheduling delivery on a specified date; means for tracking expenses to ensure that monthly expenditures stay within budget based on the stored data; means for checking the format of the input content in real time and transmitting the data if the format is determined to be correct; and means for generating low-cost alternative recipes based on expense tracking. This makes it possible for the user to easily decide on daily menus and automatically purchase and manage ingredients within budget.

[0966] A "user" is someone who uses the system to input monthly budgets, the number of people, and foods to be excluded, and then instructs the system to generate daily menus and lists of necessary ingredients.

[0967] The "means of input" refer to a system that provides an interface for users to input their monthly budget, number of people, and foods they wish to exclude.

[0968] "Means of saving" refers to a system that saves entered data to a storage device such as a database so that it can be used later.

[0969] "Methods for generating menus" refer to algorithms or programs that automatically generate menus within a budget and excluding excluded foods, based on stored data.

[0970] "Means for generating ingredient lists" refers to algorithms or programs that create a list of necessary ingredients based on the generated menu.

[0971] An "e-commerce site" is a platform for selling and buying goods online.

[0972] The "add to cart" method is a system that automates the purchase process of products on e-commerce sites based on a list of ingredients.

[0973] "Means of scheduling delivery" refers to a time management system for delivering products purchased on an e-commerce site to users at a specified date and time.

[0974] A "means of tracking expenses" is a system that monitors and manages expenses in real time based on stored data to ensure that monthly spending stays within budget.

[0975] "A means of checking the format of input content in real time" refers to a system that checks immediately whether there are any problems with the format of the data entered by the user.

[0976] "Means for generating low-cost alternative recipes" are algorithms or programs that, based on cost tracking results, suggest alternative menus that can be prepared at a lower cost if the budget is likely to be exceeded.

[0977] This invention is a system designed to streamline meal management in busy households, such as dual-income families. Specifically, it provides a process for determining daily menus and automating the purchase and delivery of ingredients within a budget.

[0978] This system's program primarily functions through the collaboration of three entities: servers, terminals, and users. The following describes the processing procedures and the hardware and software used for each entity.

[0979] User actions

[0980] Users enter their monthly budget, family size, and foods they wish to exclude through a dedicated application or web interface. For example, a user might enter "Budget: 30,000 yen, Family size: 4, Exclude nuts." The entered data is checked in real time on the terminal and sent to the server only after it is confirmed to be in the correct format.

[0981] Server Processing

[0982] The server performs the following main tasks:

[0983] 1. Data Storage and Management: Receive data submitted by users and store it in a database. For example, use a database system such as MongoDB or MySQL.

[0984] 2. Menu Generation: Based on saved data, menus are generated that stay within budget and do not include excluded foods. Specifically, a generation AI model is used to algorithmically select menus that meet the user's criteria.

[0985] 3. Generating an ingredient list: Based on the suggested menu, generate a list of necessary ingredients. For example, create a list such as, "For chicken curry, you'll need chicken, onions, curry powder, spices, etc."

[0986] 4. Integration with e-commerce sites: Based on the confirmed list of ingredients, the system integrates with the API of an e-commerce site (e.g., Yahoo! Shopping API) to add ingredients to the cart and schedule delivery dates and times.

[0987] 5. Budget Management: Track expenses in real time based on purchase history and check actual spending against the user's budget. If the budget is likely to be exceeded, generate lower-cost alternative recipes.

[0988] Terminal processing

[0989] The terminal (the device operated by the user) provides the following functions:

[0990] 1. Data Verification and Transmission: The data entered by the user is verified in real time, and if there are no formatting issues, it is sent to the server.

[0991] 2. Display of Menus and Ingredient Lists: Menu suggestions and ingredient lists received from the server are displayed in the user interface. For example, it might display "Dinner on Day 1: Chicken Curry, Day 2: Stir-fried Vegetables."

[0992] 3. User Confirmation and Modification: Provide an interface that allows users to review the ingredient list and delivery details and make modifications as needed.

[0993] 4. Displaying budget progress: Display budget progress and warning messages to allow users to always check the budget status.

[0994] Specific example

[0995] For example, a user registers the following conditions:

[0996] Monthly budget: 30,000 yen

[0997] Family size: 4 people

[0998] Excluded items: Nuts

[0999] Based on these conditions, the system performs the following steps: First, it generates a 30-day menu at the beginning of the month within the given conditions. For example, a one-week menu might be suggested as follows:

[1000] Day 1: Lunch: Sandwich, Dinner: Chicken Curry

[1001] Day 2: Lunch: Spaghetti, Dinner: Stir-fried vegetables

[1002] ...

[1003] The system then creates a list of ingredients needed for each menu item. For example, it might present a list such as, "For Day 1's chicken curry, you'll need chicken, onions, curry powder, spices, etc." The user reviews this list, makes any necessary adjustments, places an order using the e-commerce site's API, and schedules delivery for the specified day. This entire process allows users to easily decide on their daily menus and automatically purchase the necessary ingredients. The system also manages spending in real time to stay within budget, ensuring a stable diet even amid rising prices.

[1004] Example of a prompt

[1005] The following are specific examples of prompts to input into a generative AI model:

[1006] "Please create a system to manage meals for a dual-income household with a monthly budget of 30,000 yen for four people. Please provide detailed instructions, including specific menu suggestions when the user specifies nuts as a food to exclude, how to generate a list of necessary ingredients, and the process for purchasing and delivering those ingredients."

[1007] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1008] Program processing flow

[1009] Step 1: User Registration

[1010] input

[1011] User: Enter your budget, family size, and foods you want to exclude using the dedicated application or web interface.

[1012] Specific input example: "Budget 30,000 yen, family of 4, exclude nuts"

[1013] Specific actions

[1014] Terminal: Checks user input in real time. If there are formatting issues, displays an error message and prompts correction. If the format is correct, encrypts the data and sends it to the server.

[1015] output

[1016] The input data is sent to the server.

[1017] Step 2: Save Data

[1018] input

[1019] User data sent from the device

[1020] Specific actions

[1021] Server: Analyzes received data and adds it to the database as a new record. Simultaneously, it generates a user ID and stores it as user initial information. For example, it might use MongoDB or MySQL to manage the data.

[1022] output

[1023] User information is stored in the database.

[1024] Step 3: Menu Generation

[1025] input

[1026] User budget, family size, and exclusion food list retrieved from the database.

[1027] Specific actions

[1028] Server: Executes database queries to retrieve user information. Based on the retrieved data, a generative AI model is used to generate menus that stay within budget and exclude excluded foods. For example, it calculates the daily cost for four people with a monthly budget of 30,000 yen and uses an algorithm to select recipes that do not contain nuts.

[1029] output

[1030] The generated menu information is sent to the terminal.

[1031] Step 4: Menu Display

[1032] input

[1033] Menu information sent from the server

[1034] Specific actions

[1035] Terminal: Displays received menu information on the user interface. For example, it might suggest "Dinner on day 1 is chicken curry" or "Dinner on day 2 is stir-fried vegetables."

[1036] output

[1037] The user checks the menu.

[1038] Step 5: Generating the ingredient list

[1039] input

[1040] Proposed menu information

[1041] Specific actions

[1042] Server: Retrieves the necessary ingredients for each menu item from the database and creates a list. For example, it generates a list such as, "Chicken curry requires chicken, onions, curry powder, spices, etc."

[1043] output

[1044] The generated list of ingredients is sent to the terminal.

[1045] Step 6: Check and correct the ingredient list

[1046] input

[1047] Ingredient list sent from the server

[1048] Specific actions

[1049] Terminal: Displays the ingredient list in the user interface and provides buttons such as "Edit," "Delete," and "Add." When the user makes a modification, the data is sent back to the server.

[1050] output

[1051] Confirmed list of ingredients

[1052] Step 7: Purchase ingredients and set up delivery.

[1053] input

[1054] Confirmed list of ingredients

[1055] Specific actions

[1056] Server: Based on the ingredient list, it connects with the API of an e-commerce site (e.g., Yahoo! Shopping API). For example, it adds "chicken, onion, curry powder" to the cart and sets the delivery date to "Month Day".

[1057] output

[1058] The purchase process is completed on the e-commerce site, and delivery is scheduled.

[1059] Step 8: Notification of purchase details and delivery date / time

[1060] input

[1061] Purchase procedure and delivery schedule information

[1062] Specific actions

[1063] Terminal: The information is displayed as a pop-up message on the user interface, providing a screen for the user to confirm.

[1064] output

[1065] A user verification flag is returned to the server.

[1066] Step 9: Budget management and alternative recipe suggestions

[1067] input

[1068] Purchase history data

[1069] Specific actions

[1070] Server: Tracks expenses in real time and manages spending against the user's budget. If the budget is likely to be exceeded, it uses a generative AI model to suggest lower-cost alternative recipes.

[1071] output

[1072] An alternative recipe is sent to the device.

[1073] Step 10: Displaying budget progress

[1074] input

[1075] Budget progress and warning messages sent from the server

[1076] Specific actions

[1077] Terminal: The budget management screen displays information such as "Current expenses: 28,000 yen" and "Remaining budget: 2,000 yen," and a warning icon flashes to notify the user.

[1078] output

[1079] User verification and budget adjustment

[1080] (Application Example 1)

[1081] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1082] There are challenges in streamlining household meal management, such as automating daily meal planning, grocery purchasing, and delivery within budget for dual-income households. Furthermore, it's necessary to minimize the effort users have to manually check and adjust ingredients, and to provide real-time updates on budget progress and warning messages.

[1083] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1084] In this invention, the server includes means for the user to input a monthly budget, number of people, and foods to be excluded; means for storing the input data; means for generating a menu that does not include excluded foods and stays within the budget based on the stored data; means for generating a list of necessary ingredients based on the generated menu; means for adding products to an online shopping cart based on the list of necessary ingredients and scheduling delivery on a specified date; means for tracking expenses based on the stored data to ensure that monthly expenditures stay within the budget; and means for providing an interface that allows the user to check the budget progress and warning messages. This automates the daily tasks of deciding on menus, purchasing ingredients, and delivery, and enables more efficient budget management.

[1085] "A means for users to input monthly budgets, number of people, and foods to exclude" refers to an interface for inputting basic data for monthly meal planning, taking into account constraints such as family structure, budget, and allergies.

[1086] "Means of saving entered data" refers to a function that stores information entered by the user on a cloud server or database, making it accessible later.

[1087] "A means of generating a menu that stays within budget and excludes excluded foods based on saved data" refers to an algorithm that uses saved user data to automatically plan a menu that stays within budget and excludes specified foods.

[1088] "A means of generating a list of necessary ingredients based on a generated menu" refers to a function that creates a list of ingredients needed to prepare a meal according to a planned menu.

[1089] "A method for adding products to an online shopping cart based on a list of necessary ingredients and scheduling delivery on a specified date" refers to a function that uses the API of an online shopping site to add products to the cart based on the created ingredient list and arranges delivery at the date and time desired by the user.

[1090] "A means of tracking expenses to ensure monthly spending stays within budget based on saved data" refers to a feature that monitors purchase history and current spending in real time and adjusts spending to stay within budget, in order to help users manage their budget.

[1091] "Means of providing an interface that allows users to check budget progress and warning messages" refers to a function that provides a screen through an application or web browser that allows users to check budget progress and related warning messages in real time.

[1092] The "means for generating menus that also consider the seasonality and nutritional balance of ingredients, and further call external e-commerce site APIs for purchasing products" refers to an algorithm and function that generates menus that include seasonal ingredients and nutritional balance, and then executes the process of purchasing ingredients using APIs from external shopping sites.

[1093] "Means of providing an interface for users to view, add, delete, or adjust the quantity of ingredients in an ingredient list" refers to a function that provides an interactive screen for users to add or delete items and adjust quantities as needed, based on the ingredient list they have created.

[1094] "A means of inputting prompt text into a generative AI model to generate recipes and ingredient information" refers to a function that executes a process in which the user inputs specific conditions or preferences in text format, and the generative AI model generates the optimal recipe and related ingredient information.

[1095] The system of this invention provides a comprehensive approach to streamline family meal management. Specifically, it supports an interface in which the user inputs a monthly budget, the number of family members, and foods to be excluded, and stores and uses this information. Data entered via a device (such as a smartphone or PC) is stored in a cloud-based database.

[1096] The server generates menus that fit within the budget and do not include excluded foods, based on the stored data. For example, if the registered data is "monthly budget of 30,000 yen, family size of 4, excluded foods: nuts," the server will generate and recommend appropriate recipes that meet those conditions. Based on the generated menus, the server automatically generates a list of necessary ingredients. This ingredient list provides an interface that allows the user to review, add, delete, and adjust the quantities of ingredients.

[1097] Furthermore, based on the list of necessary ingredients, the server interacts with the API of an online shopping site (for example, a hypothetical e-commerce API), adds the items to the cart, and schedules delivery of the ingredients at a date and time specified by the user. This allows users to obtain the necessary ingredients without leaving their homes.

[1098] Based on the stored data, the server tracks monthly spending in real time and manages it to stay within budget. It also provides an interface where users can check budget progress and receive warning messages, ensuring that spending is always properly managed. For example, it will issue an alert if there is a possibility of exceeding the budget near the end of the month.

[1099] In addition, the server creates menus that take seasonality and nutritional balance into consideration, supporting users' health management. The generated list of ingredients automates the process of purchasing by calling APIs of external e-commerce sites. Users can easily perform these operations through the interface.

[1100] As a concrete example, here is an example of a prompt message that the user might enter:

[1101] "For the development of a food and beverage management application, please propose a 30-day menu based on the following input information: budget of 30,000 yen, family size of 4, and exclude nuts. Additionally, please generate a list of ingredients required for each menu."

[1102] The main hardware / software used in implementing this invention includes a cloud-based database, server-side programming languages ​​(e.g., Python and Flask), and libraries to facilitate communication with external APIs (e.g., Requests). This reduces user effort and enables an efficient meal management system.

[1103] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1104] Step 1:

[1105] Users enter their budget, family size, and foods they wish to exclude using an application or web interface. Once the data is entered, it is sent from the device to the server. The entered data includes monthly budget, family composition, and allergy information.

[1106] Input: Budget, number of family members, foods to exclude

[1107] Output: User data sent to the server

[1108] Step 2:

[1109] The server saves the received data to a database. Once stored in the database, the user's settings become available for subsequent processing.

[1110] Input: User data sent to the server

[1111] Output: User information stored in the database

[1112] Step 3:

[1113] The server generates a menu that fits within the budget and excludes excluded foods, based on stored user information. This process also applies an algorithm that takes into account the seasonality and nutritional balance of ingredients.

[1114] Input: User information stored in the database

[1115] Output: Generated menu

[1116] Step 4:

[1117] The server creates a list of necessary ingredients based on the generated menu. This list details the ingredients required for each menu recipe, including quantities and units.

[1118] Input: Generated menu

[1119] Output: List of required ingredients

[1120] Step 5:

[1121] The ingredient list created by the server is sent to the user's terminal, where the user can review it. The user can add, delete, or adjust the contents of the ingredient list through the interface. The user's actions are resent to the server.

[1122] Input: List of required ingredients

[1123] Output: User-reviewed and adjusted ingredient list

[1124] Step 6:

[1125] Based on the final list of ingredients, the server calls the e-commerce site's API to add the ingredients to the cart and schedule delivery for the date and time specified by the user. The results of the API calls are stored on the server and notified to the user.

[1126] Input: User-confirmed and adjusted ingredient list

[1127] Output: Purchase information and delivery schedule in e-commerce

[1128] Step 7:

[1129] The server tracks purchase history and current spending in real time, managing it to stay within budget. If the budget is likely to be exceeded, it notifies the user with alternatives or warnings.

[1130] Input: Purchase history, current spending data

[1131] Output: Spending tracking information, warning messages

[1132] Step 8:

[1133] Users can check budget progress and warning messages through the interface. The server provides this information in real time, allowing users to always be aware of their current budget status.

[1134] Input: Expense tracking information, warning messages

[1135] Output: Budget progress and warning messages displayed in the user interface.

[1136] An example of a specific prompt message is as follows:

[1137] "For the development of a food and beverage management application, please propose a 30-day menu based on the following input information: budget of 30,000 yen, family size of 4, and exclude nuts. Additionally, please generate a list of ingredients required for each menu."

[1138] The above is a description of the specific processing steps.

[1139] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1140] This invention is a system designed to streamline household meal management, particularly for dual-income households, automating the process of planning daily menus and purchasing and delivering ingredients within a budget. Furthermore, by incorporating an emotion engine that recognizes user emotions, this system provides a more personalized service by offering menu suggestions that take the user's emotional state into account.

[1141] Program execution steps

[1142] 1. User registration:

[1143] User: Open the dedicated application or web interface, enter your monthly budget, family size, and foods you want to exclude, and press the submit button. For example, enter "Budget 30,000 yen, family of 4, exclude nuts."

[1144] Terminal: Validates the entered data locally and performs error checks. If there are no problems, the data is sent to the server.

[1145] Server: Receives data sent from terminals and stores it in the database. Manages data based on user configuration information.

[1146] 2. Emotion recognition:

[1147] Terminal: When a user enters data, it collects facial expressions, voice tone, and text content.

[1148] Emotion Engine: Analyzes collected data to recognize the user's emotions. For example, it uses facial recognition and voice analysis technologies to extract emotions such as fatigue or stress.

[1149] Server: Receives the analysis results from the emotion engine and saves them to the database.

[1150] 3. Menu suggestions:

[1151] Server: Based on user configuration information and sentiment data, the server generates an optimal menu considering the user's emotional state. For example, if the server determines that the user is tired, it will suggest easy-to-prepare dishes.

[1152] Terminal: Displays menu suggestions sent from the server to the user. For example, it might suggest "Dinner on day 1: Chicken curry, Day 2: Stir-fried vegetables."

[1153] 4. Generating the ingredient list:

[1154] Server: Based on the proposed menu, it generates a list of necessary ingredients. For example, it creates a list such as, "For chicken curry, you need chicken, onions, curry powder, spices, etc."

[1155] Terminal: Provides an interface that displays a list of ingredients to the user, allowing the user to review and make necessary corrections.

[1156] 5. Grocery purchase and delivery settings:

[1157] Server: Based on the confirmed list of ingredients, the server adds items to the cart via the e-commerce site's API and schedules the delivery date and time. For example, it might set "Purchase chicken, onions, and curry powder on [Month] [Day], and deliver on [Month] [Day]."

[1158] Terminal: Displays a screen to the user to notify them of purchase details and delivery date / time, and for the user to confirm.

[1159] 6. Budget Management:

[1160] Server: Tracks expenses in real time based on purchase history. Checks actual spending against the user's budget and manages spending to stay within budget.

[1161] Terminal: Displays budget progress and warning messages to the user. For example, it might display "Current expenses are 15,000 yen, remaining budget is 15,000 yen."

[1162] Specific example

[1163] For example, a user registers the following conditions:

[1164] Monthly budget: 30,000 yen

[1165] Family size: 4 people

[1166] Excluded items: Nuts

[1167] Based on these conditions, when the user enters information today, the device collects the user's facial expressions and voice tone, and the emotion engine analyzes them. The emotion engine recognizes that the user is tired. Based on this information, the server suggests a menu that can be easily prepared: "Dinner on day 1: Chicken curry, Day 2: Stir-fried vegetables."

[1168] The system then creates a list of ingredients needed for each menu item, presenting a list such as, "For Day 1's chicken curry, you'll need chicken, onions, curry powder, spices, etc." The user reviews this list, makes any necessary adjustments, places an order using the e-commerce site's API, and schedules delivery for the specified date.

[1169] Furthermore, the system manages spending in real time to keep it within budget, displaying information such as "Current spending: ¥15,000, remaining budget: ¥15,000," allowing users to constantly monitor their budget status. This entire process enables users to effortlessly decide on daily menus and automatically purchase necessary ingredients. In addition, by combining this with an emotion engine, the system can provide optimal suggestions tailored to the user's emotional state.

[1170] The following describes the processing flow.

[1171] Step 1: The user opens the dedicated application or web interface, enters their monthly budget, number of family members, and foods they want to exclude, and presses the submit button. For example, they might enter "Budget 30,000 yen, family of 4, exclude nuts."

[1172] Step 2: The terminal locally validates the entered data and performs error checks. If there are no errors, it sends the input data to the server.

[1173] Step 3: The server receives the data sent from the terminal and saves it to the database. This saves the user's configuration information for management purposes.

[1174] Step 4: The device collects the user's facial expressions, voice tone, and entered text. This data provides the foundational information needed to understand the user's emotional state.

[1175] Step 5: The emotion engine analyzes the collected emotion data to recognize the user's emotional state. For example, it uses facial recognition and voice analysis technologies to determine whether the user is tired, stressed, etc.

[1176] Step 6: The server receives the analysis results from the emotion engine and stores them in the database. This allows for tracking and long-term management of the user's emotional state.

[1177] Step 7: The server generates a menu based on the user's settings and emotional data. It also takes the user's emotional state into consideration; for example, if the user is tired, it selects dishes that are easy to prepare.

[1178] Step 8: The server sends the generated menu data to the terminal and displays it to the user. Here, it visually presents specific details such as "Dinner on day 1 is chicken curry, and on day 2 is stir-fried vegetables."

[1179] Step 9: The user reviews the menu displayed on their device and adds or removes ingredients or adjusts quantities as needed. After reviewing, they send the changes to the server.

[1180] Step 10: Based on the final list of ingredients received from the user, the server uses the e-commerce site's API to add items to the cart. Each ingredient is added to the cart sequentially, and the purchase process is completed.

[1181] Step 11: After the server completes the purchase process, it schedules the delivery date and time based on the user's preferences. For example, it might set "Purchase chicken, onions, and curry powder on [Month] [Day], and deliver on [Month] [Day]."

[1182] Step 12: The server sends the purchase details and delivery date and time to the terminal and displays a confirmation screen to the user. The user reviews the order details and delivery schedule and makes any necessary changes.

[1183] Step 13: The server tracks costs in real time based on purchase history. It manages to prevent budget overruns and suggests alternative recipes if shortages are likely.

[1184] Step 14: The device displays budget progress and warning messages to the user. For example, it provides information such as "Current expenses are 15,000 yen, remaining budget is 15,000 yen" to ensure the user is always aware of their budget status.

[1185] This allows users to easily decide on their daily menus and efficiently purchase and have necessary ingredients delivered. Furthermore, by utilizing an emotion engine, the system can provide optimal menu suggestions tailored to the user's emotional state.

[1186] (Example 2)

[1187] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1188] This invention aims to reduce the burden on users, particularly in dual-income households, by automating the processes of daily meal planning, purchasing and delivery of ingredients within budget, and providing personalized services that take into account the user's emotional state. Furthermore, it aims to provide a system that integrates these processes, including meal suggestions that consider the season and nutritional balance, and budget management.

[1189] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1190] In this invention, the server includes means for storing user input data, means for analyzing emotional data, and means for generating menus and creating ingredient lists. This enables menu suggestions that take into account the user's emotional state, as well as automated ingredient purchasing and delivery.

[1191] "User input" refers to the process where users enter necessary information, such as monthly budgets, number of people, and foods they wish to exclude, through a dedicated application or web interface.

[1192] "Emotion recognition" is the process of recognizing a user's emotional state by analyzing their facial expressions, tone of voice, text content, and other factors.

[1193] "Menu generation" refers to the automatic creation of a menu that fits within the budget and does not exclude certain foods, based on the user's settings and sentiment data.

[1194] "Ingredient list generation" refers to creating a list of necessary ingredients based on the proposed menu.

[1195] An "e-commerce site" is an online platform for purchasing goods via the internet.

[1196] "Tracking" refers to the real-time management of expenses to ensure that a user's monthly spending stays within budget.

[1197] An "interface" is the user interface or operating environment through which a user interacts with a system, performing tasks such as inputting data, confirming information, and making corrections.

[1198] This invention is a system designed to streamline household meal management, particularly for dual-income households, automating the process of planning daily menus and purchasing and delivering ingredients within a budget. Furthermore, by incorporating an emotion engine that recognizes user emotions, this system provides a more personalized service by offering menu suggestions that take the user's emotional state into account.

[1199] User input and data storage

[1200] First, the user uses a dedicated application or web interface to enter their monthly budget, family size, and foods they wish to exclude. This data is validated and error-checked on the device before being sent to the server and stored in the database. For example, information such as "budget 30,000 yen, family of 4, exclude nuts" might be entered.

[1201] emotion recognition

[1202] When a user enters information, the device collects the user's facial expressions, voice tone, and text content. This collected data is analyzed by an emotion engine. Specifically, it uses facial recognition technology (e.g., Google Cloud Vision) and voice analysis technology (e.g., IBM Watson) to recognize the user's emotions. The analysis results are sent to a server and stored in a database.

[1203] Menu generation

[1204] The server generates an optimal menu based on saved user settings and emotional data. Considering the user's emotional state, for example, if the user is tired, a menu that is easy to prepare will be suggested. The suggested menu is sent to the terminal and displayed to the user. Specifically, a menu such as "Dinner on day 1: Chicken curry, Day 2: Stir-fried vegetables" might be suggested.

[1205] Generating and purchasing a list of ingredients.

[1206] The server generates a list of necessary ingredients based on the suggested menu. This list is displayed on the terminal for user confirmation and modification. The server then adds the items to the cart via the e-commerce site's API and schedules the delivery date and time. For example, it might be set to "Purchase chicken, onions, and curry powder on [Month] [Day], and deliver on [Month] [Day]."

[1207] Budget management and expense tracking

[1208] The server tracks purchase history in real time and checks actual spending against the user's budget. For example, it might display "Current spending: 15,000 yen, remaining budget: 15,000 yen." This allows users to constantly monitor their budget progress.

[1209] Specific example

[1210] For example, a user registers the following conditions:

[1211] Monthly budget: 30,000 yen

[1212] Family size: 4 people

[1213] Excluded items: Nuts

[1214] When you enter your information today, the device collects your facial expressions and voice tone, and an emotion engine analyzes this data. The emotion engine recognizes that the user is tired, and based on this information, the server suggests a menu that is easy to prepare. For example, it might suggest "Chicken curry for dinner on day 1, and stir-fried vegetables on day 2." A list of ingredients is also generated, for example, "For chicken curry on day 1, you'll need chicken, onions, curry powder, spices, etc." An order is then placed through an e-commerce site, and delivery is scheduled for the specified date. Finally, purchase history is tracked in real time, ensuring that your budget is properly managed.

[1215] Example of a prompt

[1216] "If a user has a monthly budget of 30,000 yen, a family of four, and wants to exclude nuts, and they are perceived as tired when deciding what to eat today, suggest chicken curry or stir-fried vegetables as the best menu for them."

[1217] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1218] Program execution steps

[1219] Step 1: User Registration

[1220] The user opens a dedicated application or web interface and enters their monthly budget, the number of family members, and the foods they want to exclude.

[1221] The terminal locally validates the input data and performs error checks. For example, it verifies that the budget and number of family members are in numerical format and that no invalid characters are included. If there are no problems, it sends the data to the server.

[1222] Input: User enters information such as "budget of 30,000 yen, family of 4, excludes nuts".

[1223] Processing: Local validation and error checking.

[1224] Output: The validated input data is sent to the server.

[1225] The server saves the received data to the database and returns a registration completion message to the user. If the entire process is successful, feedback such as "Registration Complete" is sent to the user.

[1226] Input: Validated data from the terminal.

[1227] Processing: Save data to the database.

[1228] Output: Send a registration completion message to the device.

[1229] Step 2: Emotion Recognition

[1230] The device collects facial expressions, voice tone, and text content when the user enters data.

[1231] Input: User's facial expression, voice tone, and input text.

[1232] Processing: Emotional data is collected by integrating facial expressions, voice tone, and text.

[1233] Output: Collected data is sent to the emotion engine.

[1234] The emotion engine analyzes collected data to recognize the user's emotions. Specifically, it uses facial recognition and voice analysis technologies to extract emotions such as fatigue and stress.

[1235] Input: Facial expressions, voice tone, and text data from the device.

[1236] Processing: Sentiment analysis using machine learning models.

[1237] Output: Send emotion recognition results to the server.

[1238] The server receives the analysis results from the emotion engine and stores them in the database.

[1239] Input: Analysis results from the emotion engine.

[1240] Processing: Save to database.

[1241] Output: Feedback of saved results.

[1242] Step 3: Menu Suggestions

[1243] The server generates an optimal menu based on the user's settings and emotional data. For example, if the server determines that the user is tired, it will suggest easy-to-prepare dishes.

[1244] Input: Configuration information and sentiment data.

[1245] Processing: Data calculations to generate menus by integrating configuration information and sentiment data.

[1246] Output: Sends the generated menu data to the terminal.

[1247] The terminal displays menu suggestions sent from the server to the user.

[1248] Input: Menu data from the server.

[1249] Processing: Reflecting menu data in the interface.

[1250] Output: Displays menus to the user, such as "Dinner on day 1 is chicken curry, and on day 2 is stir-fried vegetables."

[1251] Step 4: Generating the ingredient list

[1252] The server generates a list of necessary ingredients based on the proposed menu. For example, it might list "Chicken curry requires chicken, onions, curry powder, spices, etc."

[1253] Input: Suggested menu data.

[1254] Processing: Based on the menu data, retrieve a list of necessary ingredients from the database and create a list.

[1255] Output: Sends the generated ingredient list data to the terminal.

[1256] The terminal displays a list of ingredients to the user and provides an interface for the user to review and make necessary corrections.

[1257] Input: Ingredient list data from the server.

[1258] Processing: Reflecting ingredient list data in the interface.

[1259] Output: Displays a list of ingredients to the user, allowing for confirmation and modification.

[1260] Step 5: Purchase ingredients and set up delivery.

[1261] Based on the confirmed list of ingredients, the server adds items to the cart via the e-commerce site's API and schedules the delivery date and time.

[1262] Input: User-modified list of confirmed ingredients.

[1263] Processing: Add products to the cart and create a delivery schedule via the e-commerce site's API.

[1264] Output: Sends purchase and delivery details to the device.

[1265] The device notifies the user of the purchase details and delivery date and time, and displays a screen for the user to confirm.

[1266] Input: Purchase and delivery details data from the server.

[1267] Processing: Reflects purchase details and delivery schedule in the interface.

[1268] Output: Notify the user and display a confirmation screen.

[1269] Step 6: Budget Management

[1270] The server tracks expenses in real time based on purchase history. It checks actual spending against the user's budget and manages it to stay within budget.

[1271] Input: Purchase history data.

[1272] Processing: Real-time cost tracking and budget checking.

[1273] Output: Sends budget progress and warning messages to the terminal.

[1274] The terminal displays budget progress and warning messages to the user.

[1275] Input: Budget progress and warning messages from the server.

[1276] Processing: Reflect budget progress and warning messages in the interface.

[1277] Output: Displays budget progress and warning messages to the user.

[1278] (Application Example 2)

[1279] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1280] For dual-income households and busy modern people, deciding on daily menus is a significant burden, and purchasing ingredients within budget and cooking efficiently is a difficult challenge. Furthermore, failing to consider the user's emotional state during these tasks can increase stress and burden. In particular, the process of creating menus and purchasing ingredients while considering factors such as family size, food preferences, and allergy information is complex, and there is a need for an appropriate system to streamline these processes.

[1281] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for the user to input a monthly budget, number of people, and foods to be excluded; means for storing the input data; means for generating a menu that does not include excluded foods within the budget based on the stored data; means for generating a list of necessary ingredients based on the generated menu; means for adding products to a cart via an e-commerce site based on the ingredient list and scheduling delivery on a specified date; means for tracking expenses so that monthly expenditures stay within the budget based on the stored data; means for recognizing the user's emotions using facial expressions and voice tone; and means for suggesting a personalized menu based on the user's emotional data. This enables the suggestion of optimal menus with less stress according to the user's emotional state, as well as efficient purchase and management of ingredients.

[1282] A "user" is an individual or family member who uses the system.

[1283] A "budget" is the amount of money that is allowed to be spent on food and meals in a month.

[1284] "Number of people" refers to the number of family members or residents in a household who will be using the system for meal management.

[1285] "Foods to be excluded" are ingredients that users wish to avoid consuming for reasons such as health conditions, allergies, or personal preferences.

[1286] An "input method" is an interface that allows a user to provide the system with the information they need.

[1287] A "storage system" is a system that retains entered data and manages it in a way that allows it to be referenced when needed.

[1288] A "menu" is a plan that outlines the types and contents of meals a user will eat.

[1289] A "menu generation method" is a means of automatically creating a menu that does not exclude any foods, within the user's budget, based on the input data.

[1290] A "food ingredient list" is a list of ingredients needed based on the generated menu.

[1291] An "e-commerce site" is an online platform for purchasing goods via the internet.

[1292] A "delivery scheduling method" is a means of ensuring that food ingredients are delivered at the date and time specified by the user.

[1293] "Expense tracking methods" are means of monitoring and managing expenses to ensure that monthly expenditures stay within budget.

[1294] "Emotion recognition means" refers to technology that identifies emotions from a user's facial expressions and voice tone.

[1295] A "personalized menu" is an individually optimized meal plan that takes into account the user's emotional data.

[1296] This invention is a system that streamlines household meal management, automatically determining daily menus and purchasing and delivering ingredients within budget, especially for dual-income households. Furthermore, by incorporating an emotion engine that recognizes user emotions, it provides menu suggestions that take into account the user's emotional state, offering a more personalized service.

[1297] Hardware and software to be used

[1298] Smartphone: A device on which applications are installed.

[1299] Emotion recognition engine: "EmotionRecognition" analyzes facial expressions and "VoiceEmotionRecognition" analyzes voice tone.

[1300] Server: A server that maintains the database and provides APIs.

[1301] E-commerce API: An external API for purchasing groceries online.

[1302] Explanation of the process

[1303] 1. User registration:

[1304] Users input basic information such as their monthly budget, family size, and foods they wish to exclude using a dedicated application. This enables personalized meal planning and ingredient management for each household.

[1305] 2. Emotion recognition:

[1306] When a user enters information, the device collects facial expressions and voice tone, which are then analyzed by an emotion recognition engine. For example, facial recognition and voice analysis technologies can be used to extract emotional states such as whether the user is tired or stressed.

[1307] 3. Menu suggestions:

[1308] The server generates an optimal menu based on the user's settings and emotional data. For example, if it determines that the user is tired, it can suggest easy-to-prepare dishes.

[1309] 4. Generating the ingredient list:

[1310] Based on the suggested menu, a list of necessary ingredients is generated. This allows the user to efficiently prepare the required ingredients.

[1311] 5. Grocery purchase and delivery settings:

[1312] Based on the confirmed list of ingredients, the server adds items to the cart via the API of online e-commerce sites and schedules delivery dates and times. This allows users to obtain ingredients without any hassle.

[1313] 6. Budget Management:

[1314] The server tracks expenses based on purchase history to ensure they stay within budget and displays the user's current spending status. For example, it might show information such as "Current spending: ¥15,000, remaining budget: ¥15,000" in real time.

[1315] Specific example

[1316] For example, a user registers the following conditions:

[1317] Monthly budget: 30,000 yen

[1318] Family size: 4 people

[1319] Excluded items: Nuts

[1320] When a user enters information today, the device collects the user's facial expressions and voice tone, which are then analyzed by an emotion engine. The emotion engine recognizes that the user is tired. Based on this information, the server suggests easy-to-prepare meals, such as "Dinner on Day 1: Chicken Curry, Day 2: Stir-fried Vegetables." The system then creates a list of ingredients needed for each meal, presenting a list such as, "For Day 1's Chicken Curry: Chicken, onions, curry powder, spices..." The user reviews this list, makes any necessary adjustments, places an order using the e-commerce site's API, and schedules delivery for the specified day. Furthermore, the system manages spending in real time to keep it within budget, displaying information such as, "Current spending: 15,000 yen, remaining budget: 15,000 yen," allowing the user to constantly monitor their budget status.

[1321] Example of a prompt

[1322] User-entered data: Budget of 30,000 yen, family of 4, nuts excluded. Facial expressions and voice tone were analyzed to determine the user was tired. Based on this information, suggest a simple meal plan for the user: "Dinner on day 1: Chicken curry, Day 2: Stir-fried vegetables."

[1323] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1324] Step 1:

[1325] The user enters their monthly budget, number of people, and foods they want to exclude. The user opens a dedicated application and enters information about the budget, family size, and excluded foods. For example, they might enter "Budget 30,000 yen, family of 4, exclude nuts." This input data is locally validated on the terminal to check for any problems with the data format. If there are no problems, the terminal sends this data to the server. The input data is used as input, and validation and transmission are output.

[1326] Step 2:

[1327] The server receives user data sent from the terminal and stores it in the database. The stored data includes the user's budget, family size, and excluded foods. The server analyzes the data and converts it into the necessary data structure. For example, if a user enters "Exclude nuts," nuts are added to the excluded foods list. The input data is user information sent from the terminal, and the output is the information stored in the database.

[1328] Step 3:

[1329] The device collects facial expressions and voice tone when the user inputs information. Equipped with a camera and microphone, the device captures the user's facial expressions and voice data in real time. This data is sent to an emotion engine for analysis. The input is data on facial expressions and voice tone, while the output is analyzed emotion data.

[1330] Step 4:

[1331] The emotion engine analyzes the collected data and sends it to the server. For example, it uses "EmotionRecognition" for facial expression analysis and "VoiceEmotionRecognition" for voice analysis to determine if the user is tired, stressed, etc. This information is then sent to the server. The input data is the collected and analyzed emotion data, and the output is sent to the server.

[1332] Step 5:

[1333] The server generates an optimal menu based on the user's settings and emotional data. For example, if the server detects that the user is tired, it prioritizes suggesting easy-to-prepare dishes. The generated menu data is returned to the terminal and displayed to the user. The input is the user's settings and emotional data, and the output is the generated menu.

[1334] Step 6:

[1335] The server generates a list of necessary ingredients based on the proposed menu. For example, it creates a specific list such as, "For Day 1's chicken curry, you'll need chicken, onions, curry powder, spices, etc." The ingredient list is converted into a detailed data structure and managed. The input is the proposed menu, and the output is the generated ingredient list.

[1336] Step 7:

[1337] The server places an order using the e-commerce site's API based on the confirmed list of ingredients and schedules the delivery date and time. For example, it might set "Purchase chicken, onions, and curry powder on [Month] [Day], and deliver on [Month] [Day]." It calls the API, adds the ingredients to the online shopping cart, and sets the delivery date and time. The input data is the generated list of ingredients, and the output is the completed order information.

[1338] Step 8:

[1339] The server tracks expenses in real time based on purchase history and manages them to stay within budget. It periodically checks budget progress to prevent users from overspending and displays warning messages as needed. For example, it might display, "Current expenses: ¥15,000, remaining budget: ¥15,000." The input data is purchase history, and the output is budget tracking and warning messages.

[1340] Example of a prompt

[1341] User-entered data: Budget of 30,000 yen, family of 4, nuts excluded. Facial expressions and voice tone were analyzed to determine the user was tired. Based on this information, suggest a simple meal plan for the user: "Dinner on day 1: Chicken curry, Day 2: Stir-fried vegetables."

[1342] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1343] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1344] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1345] [Fourth Embodiment]

[1346] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1347] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1348] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1349] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1350] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1351] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1352] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1353] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1354] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1355] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1356] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1357] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1358] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1359] This invention is a system for streamlining household meal management, particularly for dual-income households, automating the process of planning daily menus and purchasing and delivering ingredients within a budget. The system allows users to input their monthly budget, number of people, and foods to be excluded, and then generates daily menus and lists of necessary ingredients. Furthermore, it provides a series of functions for purchasing and delivering ingredients in conjunction with e-commerce sites.

[1360] Program execution steps

[1361] 1. User registration:

[1362] User: Open the dedicated application or web interface and enter your monthly budget, family size, and foods you want to exclude. Specifically, the user might enter information such as "budget of 30,000 yen, family of 4, exclude nuts."

[1363] Terminal: Locally checks the entered data to ensure there are no formatting issues. After the check is complete, the data is sent to the server.

[1364] Server: Receives data sent from terminals, stores it in the database, and manages user-specific settings.

[1365] 2. Menu suggestions:

[1366] Server: Retrieves user configuration information from the database and generates a menu that does not include excluded foods within the budget. For example, it calculates the daily cost for four people with a budget of 30,000 yen and uses an algorithm to select recipes that do not include nuts.

[1367] Terminal: Displays menu suggestions sent from the server to the user. For example, it might display "Dinner on day 1: Chicken curry, Day 2: Stir-fried vegetables."

[1368] 3. Generating the ingredient list:

[1369] Server: Based on the proposed menu, it generates a list of necessary ingredients. For example, it creates a list such as, "For chicken curry, you need chicken, onions, curry powder, spices, etc."

[1370] Terminal: Provides an interface that displays a list of ingredients to the user, allowing the user to review and make necessary corrections.

[1371] 4. Grocery purchase and delivery settings:

[1372] Server: Based on the confirmed list of ingredients, the server connects with the API of the e-commerce site to add ingredients to the cart and schedule the delivery date and time. For example, using the Yahoo! Shopping API, it might set "Purchase chicken, onions, and curry powder on [Month] [Day], and deliver on [Month] [Day]."

[1373] Terminal: Displays a screen to the user to notify them of purchase details and delivery date / time, and for the user to confirm.

[1374] 5. Budget Management:

[1375] Server: Tracks expenses in real time based on purchase history. Checks actual spending against the user's budget and manages it to stay within budget. For example, if spending is likely to exceed the budget near the end of the month, it suggests lower-cost alternative recipes.

[1376] Terminal: Display budget progress and warning messages to the user, allowing them to constantly monitor the budget status.

[1377] Specific example

[1378] For example, a user registers the following conditions:

[1379] Monthly budget: 30,000 yen

[1380] Family size: 4 people

[1381] Excluded items: Nuts

[1382] Based on these conditions, the system performs the following steps: First, at the beginning of the month, it generates a 30-day menu that fits the conditions. For example, a one-week menu might be suggested as follows:

[1383] Day 1: Lunch: Sandwich, Dinner: Chicken Curry

[1384] Day 2: Lunch: Spaghetti, Dinner: Stir-fried vegetables

[1385] ...

[1386] The system then creates a list of ingredients needed for each menu item. For example, it might present a list such as, "For Day 1's chicken curry, you'll need chicken, onions, curry powder, spices, etc." The user reviews this list, makes any necessary adjustments, places an order using the e-commerce site's API, and schedules delivery for the specified day. This entire process allows users to easily decide on their daily menus and automatically purchase the necessary ingredients. The system also manages spending in real time to stay within budget, ensuring a stable diet even amid rising prices.

[1387] The following describes the processing flow.

[1388] Step 1: The user opens the dedicated application or web interface, enters their monthly budget, number of family members, and foods they wish to exclude, and presses the submit button.

[1389] Step 2: The terminal locally validates the entered data and performs error checks. If there are no problems with validation, the data is sent to the server.

[1390] Step 3: The server receives the data sent from the terminal and saves it to the database. For example, it saves information such as "budget of 30,000 yen, family of 4, nuts excluded."

[1391] Step 4: The server retrieves the user's settings information from the database and generates daily menus based on it. It selects menus that do not include excluded foods within the budget and saves them on the server.

[1392] Step 5: The server sends the generated menu data to the terminal and displays it to the user. For example, it might display something like, "Dinner on day 1 is chicken curry, and on day 2 it's stir-fried vegetables."

[1393] Step 6: The user reviews the menu displayed on the device and adds or removes ingredients or adjusts quantities as needed.

[1394] Step 7: The device sends the final ingredient list, which has been reviewed and corrected by the user, to the server.

[1395] Step 8: Based on the confirmed ingredient list, the server adds items to the cart via the e-commerce site's API. Add the necessary ingredients to the cart one by one, and confirm the purchase once all items are available.

[1396] Step 9: After the server completes the purchase process, it schedules the delivery date and time. The schedule takes into account the user's preferred delivery date.

[1397] Step 10: The server sends the delivery details and order contents to the terminal and displays a confirmation screen to the user. The user reviews the delivery date and time and purchase details and makes any necessary changes.

[1398] Step 11: The server tracks expenses in real time based on purchase history. Check the budget and make any necessary adjustments to prevent exceeding the budget.

[1399] Step 12: The device displays budget progress and warning messages to the user. For example, it provides information such as, "Current spending is 15,000 yen, remaining budget is 15,000 yen."

[1400] This allows users to efficiently manage grocery shopping and delivery, and plan their daily meals within their budget.

[1401] (Example 1)

[1402] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1403] Currently, for busy households, such as dual-income families, deciding on daily menus and purchasing and managing ingredients within budget is a significant burden. Furthermore, there is a need for efficient methods of purchasing ingredients while considering budget management, seasonality of ingredients, and nutritional balance. Conventional systems have found it difficult to automate and optimize all of these elements, leaving a considerable amount of effort to the user.

[1404] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1405] In this invention, the server includes means for the user to input a monthly budget, number of people, and foods to be excluded; means for storing the input data; means for generating a menu that does not include excluded foods and stays within the budget based on the stored data; means for generating a list of necessary ingredients based on the generated menu; means for adding products to a cart via an e-commerce site based on the ingredient list and scheduling delivery on a specified date; means for tracking expenses to ensure that monthly expenditures stay within budget based on the stored data; means for checking the format of the input content in real time and transmitting the data if the format is determined to be correct; and means for generating low-cost alternative recipes based on expense tracking. This makes it possible for the user to easily decide on daily menus and automatically purchase and manage ingredients within budget.

[1406] A "user" is someone who uses the system to input monthly budgets, the number of people, and foods to be excluded, and then instructs the system to generate daily menus and lists of necessary ingredients.

[1407] The "means of input" refer to a system that provides an interface for users to input their monthly budget, number of people, and foods they wish to exclude.

[1408] "Means of saving" refers to a system that saves entered data to a storage device such as a database so that it can be used later.

[1409] "Methods for generating menus" refer to algorithms or programs that automatically generate menus within a budget and excluding excluded foods, based on stored data.

[1410] "Means for generating ingredient lists" refers to algorithms or programs that create a list of necessary ingredients based on the generated menu.

[1411] An "e-commerce site" is a platform for selling and buying goods online.

[1412] The "add to cart" method is a system that automates the purchase process of products on e-commerce sites based on a list of ingredients.

[1413] "Means of scheduling delivery" refers to a time management system for delivering products purchased on an e-commerce site to users at a specified date and time.

[1414] A "means of tracking expenses" is a system that monitors and manages expenses in real time based on stored data to ensure that monthly spending stays within budget.

[1415] "A means of checking the format of input content in real time" refers to a system that checks immediately whether there are any problems with the format of the data entered by the user.

[1416] "Means for generating low-cost alternative recipes" are algorithms or programs that, based on cost tracking results, suggest alternative menus that can be prepared at a lower cost if the budget is likely to be exceeded.

[1417] This invention is a system designed to streamline meal management in busy households, such as dual-income families. Specifically, it provides a process for determining daily menus and automating the purchase and delivery of ingredients within a budget.

[1418] This system's program primarily functions through the collaboration of three entities: servers, terminals, and users. The following describes the processing procedures and the hardware and software used for each entity.

[1419] User actions

[1420] Users enter their monthly budget, family size, and foods they wish to exclude through a dedicated application or web interface. For example, a user might enter "Budget: 30,000 yen, Family size: 4, Exclude nuts." The entered data is checked in real time on the terminal and sent to the server only after it is confirmed to be in the correct format.

[1421] Server Processing

[1422] The server performs the following main tasks:

[1423] 1. Data Storage and Management: Receive data submitted by users and store it in a database. For example, use a database system such as MongoDB or MySQL.

[1424] 2. Menu Generation: Based on saved data, menus are generated that stay within budget and do not include excluded foods. Specifically, a generation AI model is used to algorithmically select menus that meet the user's criteria.

[1425] 3. Generating an ingredient list: Based on the suggested menu, generate a list of necessary ingredients. For example, create a list such as, "For chicken curry, you'll need chicken, onions, curry powder, spices, etc."

[1426] 4. Integration with e-commerce sites: Based on the confirmed list of ingredients, the system integrates with the API of an e-commerce site (e.g., Yahoo! Shopping API) to add ingredients to the cart and schedule delivery dates and times.

[1427] 5. Budget Management: Track expenses in real time based on purchase history and check actual spending against the user's budget. If the budget is likely to be exceeded, generate lower-cost alternative recipes.

[1428] Terminal processing

[1429] The terminal (the device operated by the user) provides the following functions:

[1430] 1. Data Verification and Transmission: The data entered by the user is verified in real time, and if there are no formatting issues, it is sent to the server.

[1431] 2. Display of Menus and Ingredient Lists: Menu suggestions and ingredient lists received from the server are displayed in the user interface. For example, it might display "Dinner on Day 1: Chicken Curry, Day 2: Stir-fried Vegetables."

[1432] 3. User Confirmation and Modification: Provide an interface that allows users to review the ingredient list and delivery details and make modifications as needed.

[1433] 4. Displaying budget progress: Display budget progress and warning messages to allow users to always check the budget status.

[1434] Specific example

[1435] For example, a user registers the following conditions:

[1436] Monthly budget: 30,000 yen

[1437] Family size: 4 people

[1438] Excluded items: Nuts

[1439] Based on these conditions, the system performs the following steps: First, it generates a 30-day menu at the beginning of the month within the given conditions. For example, a one-week menu might be suggested as follows:

[1440] Day 1: Lunch: Sandwich, Dinner: Chicken Curry

[1441] Day 2: Lunch: Spaghetti, Dinner: Stir-fried vegetables

[1442] ...

[1443] The system then creates a list of ingredients needed for each menu item. For example, it might present a list such as, "For Day 1's chicken curry, you'll need chicken, onions, curry powder, spices, etc." The user reviews this list, makes any necessary adjustments, places an order using the e-commerce site's API, and schedules delivery for the specified day. This entire process allows users to easily decide on their daily menus and automatically purchase the necessary ingredients. The system also manages spending in real time to stay within budget, ensuring a stable diet even amid rising prices.

[1444] Example of a prompt

[1445] The following are specific examples of prompts to input into a generative AI model:

[1446] "Please create a system to manage meals for a dual-income household with a monthly budget of 30,000 yen for four people. Please provide detailed instructions, including specific menu suggestions when the user specifies nuts as a food to exclude, how to generate a list of necessary ingredients, and the process for purchasing and delivering those ingredients."

[1447] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1448] Program processing flow

[1449] Step 1: User Registration

[1450] input

[1451] User: Enter your budget, family size, and foods you want to exclude using the dedicated application or web interface.

[1452] Specific input example: "Budget 30,000 yen, family of 4, exclude nuts"

[1453] Specific actions

[1454] Terminal: Checks user input in real time. If there are formatting issues, displays an error message and prompts correction. If the format is correct, encrypts the data and sends it to the server.

[1455] output

[1456] The input data is sent to the server.

[1457] Step 2: Save Data

[1458] input

[1459] User data sent from the device

[1460] Specific actions

[1461] Server: Analyzes received data and adds it to the database as a new record. Simultaneously, it generates a user ID and stores it as user initial information. For example, it might use MongoDB or MySQL to manage the data.

[1462] output

[1463] User information is stored in the database.

[1464] Step 3: Menu Generation

[1465] input

[1466] User budget, family size, and exclusion food list retrieved from the database.

[1467] Specific actions

[1468] Server: Executes database queries to retrieve user information. Based on the retrieved data, a generative AI model is used to generate menus that stay within budget and exclude excluded foods. For example, it calculates the daily cost for four people with a monthly budget of 30,000 yen and uses an algorithm to select recipes that do not contain nuts.

[1469] output

[1470] The generated menu information is sent to the terminal.

[1471] Step 4: Menu Display

[1472] input

[1473] Menu information sent from the server

[1474] Specific actions

[1475] Terminal: Displays received menu information on the user interface. For example, it might suggest "Dinner on day 1 is chicken curry" or "Dinner on day 2 is stir-fried vegetables."

[1476] output

[1477] The user checks the menu.

[1478] Step 5: Generating the ingredient list

[1479] input

[1480] Proposed menu information

[1481] Specific actions

[1482] Server: Retrieves the necessary ingredients for each menu item from the database and creates a list. For example, it generates a list such as, "Chicken curry requires chicken, onions, curry powder, spices, etc."

[1483] output

[1484] The generated list of ingredients is sent to the terminal.

[1485] Step 6: Check and correct the ingredient list

[1486] input

[1487] Ingredient list sent from the server

[1488] Specific actions

[1489] Terminal: Displays the ingredient list in the user interface and provides buttons such as "Edit," "Delete," and "Add." When the user makes a modification, the data is sent back to the server.

[1490] output

[1491] Confirmed list of ingredients

[1492] Step 7: Purchase ingredients and set up delivery.

[1493] input

[1494] Confirmed list of ingredients

[1495] Specific actions

[1496] Server: Based on the ingredient list, it connects with the API of an e-commerce site (e.g., Yahoo! Shopping API). For example, it adds "chicken, onion, curry powder" to the cart and sets the delivery date to "Month Day".

[1497] output

[1498] The purchase process is completed on the e-commerce site, and delivery is scheduled.

[1499] Step 8: Notification of purchase details and delivery date / time

[1500] input

[1501] Purchase procedure and delivery schedule information

[1502] Specific actions

[1503] Terminal: The information is displayed as a pop-up message on the user interface, providing a screen for the user to confirm.

[1504] output

[1505] A user verification flag is returned to the server.

[1506] Step 9: Budget management and alternative recipe suggestions

[1507] input

[1508] Purchase history data

[1509] Specific actions

[1510] Server: Tracks expenses in real time and manages spending against the user's budget. If the budget is likely to be exceeded, it uses a generative AI model to suggest lower-cost alternative recipes.

[1511] output

[1512] An alternative recipe is sent to the device.

[1513] Step 10: Displaying budget progress

[1514] input

[1515] Budget progress and warning messages sent from the server

[1516] Specific actions

[1517] Terminal: The budget management screen displays information such as "Current expenses: 28,000 yen" and "Remaining budget: 2,000 yen," and a warning icon flashes to notify the user.

[1518] output

[1519] User verification and budget adjustment

[1520] (Application Example 1)

[1521] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1522] There are challenges in streamlining household meal management, such as automating daily meal planning, grocery purchasing, and delivery within budget for dual-income households. Furthermore, it's necessary to minimize the effort users have to manually check and adjust ingredients, and to provide real-time updates on budget progress and warning messages.

[1523] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1524] In this invention, the server includes means for the user to input a monthly budget, number of people, and foods to be excluded; means for storing the input data; means for generating a menu that does not include excluded foods and stays within the budget based on the stored data; means for generating a list of necessary ingredients based on the generated menu; means for adding products to an online shopping cart based on the list of necessary ingredients and scheduling delivery on a specified date; means for tracking expenses based on the stored data to ensure that monthly expenditures stay within the budget; and means for providing an interface that allows the user to check the budget progress and warning messages. This automates the daily tasks of deciding on menus, purchasing ingredients, and delivery, and enables more efficient budget management.

[1525] "A means for users to input monthly budgets, number of people, and foods to exclude" refers to an interface for inputting basic data for monthly meal planning, taking into account constraints such as family structure, budget, and allergies.

[1526] "Means of saving entered data" refers to a function that stores information entered by the user on a cloud server or database, making it accessible later.

[1527] "A means of generating a menu that stays within budget and excludes excluded foods based on saved data" refers to an algorithm that uses saved user data to automatically plan a menu that stays within budget and excludes specified foods.

[1528] "A means of generating a list of necessary ingredients based on a generated menu" refers to a function that creates a list of ingredients needed to prepare a meal according to a planned menu.

[1529] "A method for adding products to an online shopping cart based on a list of necessary ingredients and scheduling delivery on a specified date" refers to a function that uses the API of an online shopping site to add products to the cart based on the created ingredient list and arranges delivery at the date and time desired by the user.

[1530] "A means of tracking expenses to ensure monthly spending stays within budget based on saved data" refers to a feature that monitors purchase history and current spending in real time and adjusts spending to stay within budget, in order to help users manage their budget.

[1531] "Means of providing an interface that allows users to check budget progress and warning messages" refers to a function that provides a screen through an application or web browser that allows users to check budget progress and related warning messages in real time.

[1532] The "means for generating menus that also consider the seasonality and nutritional balance of ingredients, and further call external e-commerce site APIs for purchasing products" refers to an algorithm and function that generates menus that include seasonal ingredients and nutritional balance, and then executes the process of purchasing ingredients using APIs from external shopping sites.

[1533] "Means of providing an interface for users to view, add, delete, or adjust the quantity of ingredients in an ingredient list" refers to a function that provides an interactive screen for users to add or delete items and adjust quantities as needed, based on the ingredient list they have created.

[1534] "A means of inputting prompt text into a generative AI model to generate recipes and ingredient information" refers to a function that executes a process in which the user inputs specific conditions or preferences in text format, and the generative AI model generates the optimal recipe and related ingredient information.

[1535] The system of this invention provides a comprehensive approach to streamline family meal management. Specifically, it supports an interface in which the user inputs a monthly budget, the number of family members, and foods to be excluded, and stores and uses this information. Data entered via a device (such as a smartphone or PC) is stored in a cloud-based database.

[1536] The server generates menus that fit within the budget and do not include excluded foods, based on the stored data. For example, if the registered data is "monthly budget of 30,000 yen, family size of 4, excluded foods: nuts," the server will generate and recommend appropriate recipes that meet those conditions. Based on the generated menus, the server automatically generates a list of necessary ingredients. This ingredient list provides an interface that allows the user to review, add, delete, and adjust the quantities of ingredients.

[1537] Furthermore, based on the list of necessary ingredients, the server interacts with the API of an online shopping site (for example, a hypothetical e-commerce API), adds the items to the cart, and schedules delivery of the ingredients at a date and time specified by the user. This allows users to obtain the necessary ingredients without leaving their homes.

[1538] Based on the stored data, the server tracks monthly spending in real time and manages it to stay within budget. It also provides an interface where users can check budget progress and receive warning messages, ensuring that spending is always properly managed. For example, it will issue an alert if there is a possibility of exceeding the budget near the end of the month.

[1539] In addition, the server creates menus that take seasonality and nutritional balance into consideration, supporting users' health management. The generated list of ingredients automates the process of purchasing by calling APIs of external e-commerce sites. Users can easily perform these operations through the interface.

[1540] As a concrete example, here is an example of a prompt message that the user might enter:

[1541] "For the development of a food and beverage management application, please propose a 30-day menu based on the following input information: budget of 30,000 yen, family size of 4, and exclude nuts. Additionally, please generate a list of ingredients required for each menu."

[1542] The main hardware / software used in implementing this invention includes a cloud-based database, server-side programming languages ​​(e.g., Python and Flask), and libraries to facilitate communication with external APIs (e.g., Requests). This reduces user effort and enables an efficient meal management system.

[1543] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1544] Step 1:

[1545] Users enter their budget, family size, and foods they wish to exclude using an application or web interface. Once the data is entered, it is sent from the device to the server. The entered data includes monthly budget, family composition, and allergy information.

[1546] Input: Budget, number of family members, foods to exclude

[1547] Output: User data sent to the server

[1548] Step 2:

[1549] The server saves the received data to a database. Once stored in the database, the user's settings become available for subsequent processing.

[1550] Input: User data sent to the server

[1551] Output: User information stored in the database

[1552] Step 3:

[1553] The server generates a menu that fits within the budget and excludes excluded foods, based on stored user information. This process also applies an algorithm that takes into account the seasonality and nutritional balance of ingredients.

[1554] Input: User information stored in the database

[1555] Output: Generated menu

[1556] Step 4:

[1557] The server creates a list of necessary ingredients based on the generated menu. This list details the ingredients required for each menu recipe, including quantities and units.

[1558] Input: Generated menu

[1559] Output: List of required ingredients

[1560] Step 5:

[1561] The ingredient list created by the server is sent to the user's terminal, where the user can review it. The user can add, delete, or adjust the contents of the ingredient list through the interface. The user's actions are resent to the server.

[1562] Input: List of required ingredients

[1563] Output: User-reviewed and adjusted ingredient list

[1564] Step 6:

[1565] Based on the final list of ingredients, the server calls the e-commerce site's API to add the ingredients to the cart and schedule delivery for the date and time specified by the user. The results of the API calls are stored on the server and notified to the user.

[1566] Input: User-confirmed and adjusted ingredient list

[1567] Output: Purchase information and delivery schedule in e-commerce

[1568] Step 7:

[1569] The server tracks purchase history and current spending in real time, managing it to stay within budget. If the budget is likely to be exceeded, it notifies the user with alternatives or warnings.

[1570] Input: Purchase history, current spending data

[1571] Output: Spending tracking information, warning messages

[1572] Step 8:

[1573] Users can check budget progress and warning messages through the interface. The server provides this information in real time, allowing users to always be aware of their current budget status.

[1574] Input: Expense tracking information, warning messages

[1575] Output: Budget progress and warning messages displayed in the user interface.

[1576] An example of a specific prompt message is as follows:

[1577] "For the development of a food and beverage management application, please propose a 30-day menu based on the following input information: budget of 30,000 yen, family size of 4, and exclude nuts. Additionally, please generate a list of ingredients required for each menu."

[1578] The above is a description of the specific processing steps.

[1579] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1580] This invention is a system designed to streamline household meal management, particularly for dual-income households, automating the process of planning daily menus and purchasing and delivering ingredients within a budget. Furthermore, by incorporating an emotion engine that recognizes user emotions, this system provides a more personalized service by offering menu suggestions that take the user's emotional state into account.

[1581] Program execution steps

[1582] 1. User registration:

[1583] User: Open the dedicated application or web interface, enter your monthly budget, family size, and foods you want to exclude, and press the submit button. For example, enter "Budget 30,000 yen, family of 4, exclude nuts."

[1584] Terminal: Validates the entered data locally and performs error checks. If there are no problems, the data is sent to the server.

[1585] Server: Receives data sent from terminals and stores it in the database. Manages data based on user configuration information.

[1586] 2. Emotion recognition:

[1587] Terminal: When a user enters data, it collects facial expressions, voice tone, and text content.

[1588] Emotion Engine: Analyzes collected data to recognize the user's emotions. For example, it uses facial recognition and voice analysis technologies to extract emotions such as fatigue or stress.

[1589] Server: Receives the analysis results from the emotion engine and saves them to the database.

[1590] 3. Menu suggestions:

[1591] Server: Based on user configuration information and sentiment data, the server generates an optimal menu considering the user's emotional state. For example, if the server determines that the user is tired, it will suggest easy-to-prepare dishes.

[1592] Terminal: Displays menu suggestions sent from the server to the user. For example, it might suggest "Dinner on day 1: Chicken curry, Day 2: Stir-fried vegetables."

[1593] 4. Generating the ingredient list:

[1594] Server: Based on the proposed menu, it generates a list of necessary ingredients. For example, it creates a list such as, "For chicken curry, you need chicken, onions, curry powder, spices, etc."

[1595] Terminal: Provides an interface that displays a list of ingredients to the user, allowing the user to review and make necessary corrections.

[1596] 5. Grocery purchase and delivery settings:

[1597] Server: Based on the confirmed list of ingredients, the server adds items to the cart via the e-commerce site's API and schedules the delivery date and time. For example, it might set "Purchase chicken, onions, and curry powder on [Month] [Day], and deliver on [Month] [Day]."

[1598] Terminal: Displays a screen to the user to notify them of purchase details and delivery date / time, and for the user to confirm.

[1599] 6. Budget Management:

[1600] Server: Tracks expenses in real time based on purchase history. Checks actual spending against the user's budget and manages spending to stay within budget.

[1601] Terminal: Displays budget progress and warning messages to the user. For example, it might display "Current expenses are 15,000 yen, remaining budget is 15,000 yen."

[1602] Specific example

[1603] For example, a user registers the following conditions:

[1604] Monthly budget: 30,000 yen

[1605] Family size: 4 people

[1606] Excluded items: Nuts

[1607] Based on these conditions, when the user enters information today, the device collects the user's facial expressions and voice tone, and the emotion engine analyzes them. The emotion engine recognizes that the user is tired. Based on this information, the server suggests a menu that can be easily prepared: "Dinner on day 1: Chicken curry, Day 2: Stir-fried vegetables."

[1608] The system then creates a list of ingredients needed for each menu item, presenting a list such as, "For Day 1's chicken curry, you'll need chicken, onions, curry powder, spices, etc." The user reviews this list, makes any necessary adjustments, places an order using the e-commerce site's API, and schedules delivery for the specified date.

[1609] Furthermore, the system manages spending in real time to keep it within budget, displaying information such as "Current spending: ¥15,000, remaining budget: ¥15,000," allowing users to constantly monitor their budget status. This entire process enables users to effortlessly decide on daily menus and automatically purchase necessary ingredients. In addition, by combining this with an emotion engine, the system can provide optimal suggestions tailored to the user's emotional state.

[1610] The following describes the processing flow.

[1611] Step 1: The user opens the dedicated application or web interface, enters their monthly budget, number of family members, and foods they want to exclude, and presses the submit button. For example, they might enter "Budget 30,000 yen, family of 4, exclude nuts."

[1612] Step 2: The terminal locally validates the entered data and performs error checks. If there are no errors, it sends the input data to the server.

[1613] Step 3: The server receives the data sent from the terminal and saves it to the database. This saves the user's configuration information for management purposes.

[1614] Step 4: The device collects the user's facial expressions, voice tone, and entered text. This data provides the foundational information needed to understand the user's emotional state.

[1615] Step 5: The emotion engine analyzes the collected emotion data to recognize the user's emotional state. For example, it uses facial recognition and voice analysis technologies to determine whether the user is tired, stressed, etc.

[1616] Step 6: The server receives the analysis results from the emotion engine and stores them in the database. This allows for tracking and long-term management of the user's emotional state.

[1617] Step 7: The server generates a menu based on the user's settings and emotional data. It also takes the user's emotional state into consideration; for example, if the user is tired, it selects dishes that are easy to prepare.

[1618] Step 8: The server sends the generated menu data to the terminal and displays it to the user. Here, it visually presents specific details such as "Dinner on day 1 is chicken curry, and on day 2 is stir-fried vegetables."

[1619] Step 9: The user reviews the menu displayed on their device and adds or removes ingredients or adjusts quantities as needed. After reviewing, they send the changes to the server.

[1620] Step 10: Based on the final list of ingredients received from the user, the server uses the e-commerce site's API to add items to the cart. Each ingredient is added to the cart sequentially, and the purchase process is completed.

[1621] Step 11: After the server completes the purchase process, it schedules the delivery date and time based on the user's preferences. For example, it might set "Purchase chicken, onions, and curry powder on [Month] [Day], and deliver on [Month] [Day]."

[1622] Step 12: The server sends the purchase details and delivery date and time to the terminal and displays a confirmation screen to the user. The user reviews the order details and delivery schedule and makes any necessary changes.

[1623] Step 13: The server tracks costs in real time based on purchase history. It manages to prevent budget overruns and suggests alternative recipes if shortages are likely.

[1624] Step 14: The device displays budget progress and warning messages to the user. For example, it provides information such as "Current expenses are 15,000 yen, remaining budget is 15,000 yen" to ensure the user is always aware of their budget status.

[1625] This allows users to easily decide on their daily menus and efficiently purchase and have necessary ingredients delivered. Furthermore, by utilizing an emotion engine, the system can provide optimal menu suggestions tailored to the user's emotional state.

[1626] (Example 2)

[1627] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1628] This invention aims to reduce the burden on users, particularly in dual-income households, by automating the processes of daily meal planning, purchasing and delivery of ingredients within budget, and providing personalized services that take into account the user's emotional state. Furthermore, it aims to provide a system that integrates these processes, including meal suggestions that consider the season and nutritional balance, and budget management.

[1629] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1630] In this invention, the server includes means for storing user input data, means for analyzing emotional data, and means for generating menus and creating ingredient lists. This enables menu suggestions that take into account the user's emotional state, as well as automated ingredient purchasing and delivery.

[1631] "User input" refers to the process where users enter necessary information, such as monthly budgets, number of people, and foods they wish to exclude, through a dedicated application or web interface.

[1632] "Emotion recognition" is the process of recognizing a user's emotional state by analyzing their facial expressions, tone of voice, text content, and other factors.

[1633] "Menu generation" refers to the automatic creation of a menu that fits within the budget and does not exclude certain foods, based on the user's settings and sentiment data.

[1634] "Ingredient list generation" refers to creating a list of necessary ingredients based on the proposed menu.

[1635] An "e-commerce site" is an online platform for purchasing goods via the internet.

[1636] "Tracking" refers to the real-time management of expenses to ensure that a user's monthly spending stays within budget.

[1637] An "interface" is the user interface or operating environment through which a user interacts with a system, performing tasks such as inputting data, confirming information, and making corrections.

[1638] This invention is a system designed to streamline household meal management, particularly for dual-income households, automating the process of planning daily menus and purchasing and delivering ingredients within a budget. Furthermore, by incorporating an emotion engine that recognizes user emotions, this system provides a more personalized service by offering menu suggestions that take the user's emotional state into account.

[1639] User input and data storage

[1640] First, the user uses a dedicated application or web interface to enter their monthly budget, family size, and foods they wish to exclude. This data is validated and error-checked on the device before being sent to the server and stored in the database. For example, information such as "budget 30,000 yen, family of 4, exclude nuts" might be entered.

[1641] emotion recognition

[1642] When a user enters information, the device collects the user's facial expressions, voice tone, and text content. This collected data is analyzed by an emotion engine. Specifically, it uses facial recognition technology (e.g., Google Cloud Vision) and voice analysis technology (e.g., IBM Watson) to recognize the user's emotions. The analysis results are sent to a server and stored in a database.

[1643] Menu generation

[1644] The server generates an optimal menu based on saved user settings and emotional data. Considering the user's emotional state, for example, if the user is tired, a menu that is easy to prepare will be suggested. The suggested menu is sent to the terminal and displayed to the user. Specifically, a menu such as "Dinner on day 1: Chicken curry, Day 2: Stir-fried vegetables" might be suggested.

[1645] Generating and purchasing a list of ingredients.

[1646] The server generates a list of necessary ingredients based on the suggested menu. This list is displayed on the terminal for user confirmation and modification. The server then adds the items to the cart via the e-commerce site's API and schedules the delivery date and time. For example, it might be set to "Purchase chicken, onions, and curry powder on [Month] [Day], and deliver on [Month] [Day]."

[1647] Budget management and expense tracking

[1648] The server tracks purchase history in real time and checks actual spending against the user's budget. For example, it might display "Current spending: 15,000 yen, remaining budget: 15,000 yen." This allows users to constantly monitor their budget progress.

[1649] Specific example

[1650] For example, a user registers the following conditions:

[1651] Monthly budget: 30,000 yen

[1652] Family size: 4 people

[1653] Excluded items: Nuts

[1654] When you enter your information today, the device collects your facial expressions and voice tone, and an emotion engine analyzes this data. The emotion engine recognizes that the user is tired, and based on this information, the server suggests a menu that is easy to prepare. For example, it might suggest "Chicken curry for dinner on day 1, and stir-fried vegetables on day 2." A list of ingredients is also generated, for example, "For chicken curry on day 1, you'll need chicken, onions, curry powder, spices, etc." An order is then placed through an e-commerce site, and delivery is scheduled for the specified date. Finally, purchase history is tracked in real time, ensuring that your budget is properly managed.

[1655] Example of a prompt

[1656] "If a user has a monthly budget of 30,000 yen, a family of four, and wants to exclude nuts, and they are perceived as tired when deciding what to eat today, suggest chicken curry or stir-fried vegetables as the best menu for them."

[1657] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1658] Program execution steps

[1659] Step 1: User Registration

[1660] The user opens a dedicated application or web interface and enters their monthly budget, the number of family members, and the foods they want to exclude.

[1661] The terminal locally validates the input data and performs error checks. For example, it verifies that the budget and number of family members are in numerical format and that no invalid characters are included. If there are no problems, it sends the data to the server.

[1662] Input: User enters information such as "budget of 30,000 yen, family of 4, excludes nuts".

[1663] Processing: Local validation and error checking.

[1664] Output: The validated input data is sent to the server.

[1665] The server saves the received data to the database and returns a registration completion message to the user. If the entire process is successful, feedback such as "Registration Complete" is sent to the user.

[1666] Input: Validated data from the terminal.

[1667] Processing: Save data to the database.

[1668] Output: Send a registration completion message to the device.

[1669] Step 2: Emotion Recognition

[1670] The device collects facial expressions, voice tone, and text content when the user enters data.

[1671] Input: User's facial expression, voice tone, and input text.

[1672] Processing: Emotional data is collected by integrating facial expressions, voice tone, and text.

[1673] Output: Collected data is sent to the emotion engine.

[1674] The emotion engine analyzes collected data to recognize the user's emotions. Specifically, it uses facial recognition and voice analysis technologies to extract emotions such as fatigue and stress.

[1675] Input: Facial expressions, voice tone, and text data from the device.

[1676] Processing: Sentiment analysis using machine learning models.

[1677] Output: Send emotion recognition results to the server.

[1678] The server receives the analysis results from the emotion engine and stores them in the database.

[1679] Input: Analysis results from the emotion engine.

[1680] Processing: Save to database.

[1681] Output: Feedback of saved results.

[1682] Step 3: Menu Suggestions

[1683] The server generates an optimal menu based on the user's settings and emotional data. For example, if the server determines that the user is tired, it will suggest easy-to-prepare dishes.

[1684] Input: Configuration information and sentiment data.

[1685] Processing: Data calculations to generate menus by integrating configuration information and sentiment data.

[1686] Output: Sends the generated menu data to the terminal.

[1687] The terminal displays menu suggestions sent from the server to the user.

[1688] Input: Menu data from the server.

[1689] Processing: Reflecting menu data in the interface.

[1690] Output: Displays menus to the user, such as "Dinner on day 1 is chicken curry, and on day 2 is stir-fried vegetables."

[1691] Step 4: Generating the ingredient list

[1692] The server generates a list of necessary ingredients based on the proposed menu. For example, it might list "Chicken curry requires chicken, onions, curry powder, spices, etc."

[1693] Input: Suggested menu data.

[1694] Processing: Based on the menu data, retrieve a list of necessary ingredients from the database and create a list.

[1695] Output: Sends the generated ingredient list data to the terminal.

[1696] The terminal displays a list of ingredients to the user and provides an interface for the user to review and make necessary corrections.

[1697] Input: Ingredient list data from the server.

[1698] Processing: Reflecting ingredient list data in the interface.

[1699] Output: Displays a list of ingredients to the user, allowing for confirmation and modification.

[1700] Step 5: Purchase ingredients and set up delivery.

[1701] Based on the confirmed list of ingredients, the server adds items to the cart via the e-commerce site's API and schedules the delivery date and time.

[1702] Input: User-modified list of confirmed ingredients.

[1703] Processing: Add products to the cart and create a delivery schedule via the e-commerce site's API.

[1704] Output: Sends purchase and delivery details to the device.

[1705] The device notifies the user of the purchase details and delivery date and time, and displays a screen for the user to confirm.

[1706] Input: Purchase and delivery details data from the server.

[1707] Processing: Reflects purchase details and delivery schedule in the interface.

[1708] Output: Notify the user and display a confirmation screen.

[1709] Step 6: Budget Management

[1710] The server tracks expenses in real time based on purchase history. It checks actual spending against the user's budget and manages it to stay within budget.

[1711] Input: Purchase history data.

[1712] Processing: Real-time cost tracking and budget checking.

[1713] Output: Sends budget progress and warning messages to the terminal.

[1714] The terminal displays budget progress and warning messages to the user.

[1715] Input: Budget progress and warning messages from the server.

[1716] Processing: Reflect budget progress and warning messages in the interface.

[1717] Output: Displays budget progress and warning messages to the user.

[1718] (Application Example 2)

[1719] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1720] For dual-income households and busy modern people, deciding on daily menus is a significant burden, and purchasing ingredients within budget and cooking efficiently is a difficult challenge. Furthermore, failing to consider the user's emotional state during these tasks can increase stress and burden. In particular, the process of creating menus and purchasing ingredients while considering factors such as family size, food preferences, and allergy information is complex, and there is a need for an appropriate system to streamline these processes.

[1721] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for the user to input a monthly budget, number of people, and foods to be excluded; means for storing the input data; means for generating a menu that does not include excluded foods within the budget based on the stored data; means for generating a list of necessary ingredients based on the generated menu; means for adding products to a cart via an e-commerce site based on the ingredient list and scheduling delivery on a specified date; means for tracking expenses so that monthly expenditures stay within the budget based on the stored data; means for recognizing the user's emotions using facial expressions and voice tone; and means for suggesting a personalized menu based on the user's emotional data. This enables the suggestion of optimal menus with less stress according to the user's emotional state, as well as efficient purchase and management of ingredients.

[1722] A "user" is an individual or family member who uses the system.

[1723] A "budget" is the amount of money that is allowed to be spent on food and meals in a month.

[1724] "Number of people" refers to the number of family members or residents in a household who will be using the system for meal management.

[1725] "Foods to be excluded" are ingredients that users wish to avoid consuming for reasons such as health conditions, allergies, or personal preferences.

[1726] An "input method" is an interface that allows a user to provide the system with the information they need.

[1727] A "storage system" is a system that retains entered data and manages it in a way that allows it to be referenced when needed.

[1728] A "menu" is a plan that outlines the types and contents of meals a user will eat.

[1729] A "menu generation method" is a means of automatically creating a menu that does not exclude any foods, within the user's budget, based on the input data.

[1730] A "food ingredient list" is a list of ingredients needed based on the generated menu.

[1731] An "e-commerce site" is an online platform for purchasing goods via the internet.

[1732] A "delivery scheduling method" is a means of ensuring that food ingredients are delivered at the date and time specified by the user.

[1733] "Expense tracking methods" are means of monitoring and managing expenses to ensure that monthly expenditures stay within budget.

[1734] "Emotion recognition means" refers to technology that identifies emotions from a user's facial expressions and voice tone.

[1735] A "personalized menu" is an individually optimized meal plan that takes into account the user's emotional data.

[1736] This invention is a system that streamlines household meal management, automatically determining daily menus and purchasing and delivering ingredients within budget, especially for dual-income households. Furthermore, by incorporating an emotion engine that recognizes user emotions, it provides menu suggestions that take into account the user's emotional state, offering a more personalized service.

[1737] Hardware and software to be used

[1738] Smartphone: A device on which applications are installed.

[1739] Emotion recognition engine: "EmotionRecognition" analyzes facial expressions and "VoiceEmotionRecognition" analyzes voice tone.

[1740] Server: A server that maintains the database and provides APIs.

[1741] E-commerce API: An external API for purchasing groceries online.

[1742] Explanation of the process

[1743] 1. User registration:

[1744] Users input basic information such as their monthly budget, family size, and foods they wish to exclude using a dedicated application. This enables personalized meal planning and ingredient management for each household.

[1745] 2. Emotion recognition:

[1746] When a user enters information, the device collects facial expressions and voice tone, which are then analyzed by an emotion recognition engine. For example, facial recognition and voice analysis technologies can be used to extract emotional states such as whether the user is tired or stressed.

[1747] 3. Menu suggestions:

[1748] The server generates an optimal menu based on the user's settings and emotional data. For example, if it determines that the user is tired, it can suggest easy-to-prepare dishes.

[1749] 4. Generating the ingredient list:

[1750] Based on the suggested menu, a list of necessary ingredients is generated. This allows the user to efficiently prepare the required ingredients.

[1751] 5. Grocery purchase and delivery settings:

[1752] Based on the confirmed list of ingredients, the server adds items to the cart via the API of online e-commerce sites and schedules delivery dates and times. This allows users to obtain ingredients without any hassle.

[1753] 6. Budget Management:

[1754] The server tracks expenses based on purchase history to ensure they stay within budget and displays the user's current spending status. For example, it might show information such as "Current spending: ¥15,000, remaining budget: ¥15,000" in real time.

[1755] Specific example

[1756] For example, a user registers the following conditions:

[1757] Monthly budget: 30,000 yen

[1758] Family size: 4 people

[1759] Excluded items: Nuts

[1760] When a user enters information today, the device collects the user's facial expressions and voice tone, which are then analyzed by an emotion engine. The emotion engine recognizes that the user is tired. Based on this information, the server suggests easy-to-prepare meals, such as "Dinner on Day 1: Chicken Curry, Day 2: Stir-fried Vegetables." The system then creates a list of ingredients needed for each meal, presenting a list such as, "For Day 1's Chicken Curry: Chicken, onions, curry powder, spices..." The user reviews this list, makes any necessary adjustments, places an order using the e-commerce site's API, and schedules delivery for the specified day. Furthermore, the system manages spending in real time to keep it within budget, displaying information such as, "Current spending: 15,000 yen, remaining budget: 15,000 yen," allowing the user to constantly monitor their budget status.

[1761] Example of a prompt

[1762] User-entered data: Budget of 30,000 yen, family of 4, nuts excluded. Facial expressions and voice tone were analyzed to determine the user was tired. Based on this information, suggest a simple meal plan for the user: "Dinner on day 1: Chicken curry, Day 2: Stir-fried vegetables."

[1763] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1764] Step 1:

[1765] The user enters their monthly budget, number of people, and foods they want to exclude. The user opens a dedicated application and enters information about the budget, family size, and excluded foods. For example, they might enter "Budget 30,000 yen, family of 4, exclude nuts." This input data is locally validated on the terminal to check for any problems with the data format. If there are no problems, the terminal sends this data to the server. The input data is used as input, and validation and transmission are output.

[1766] Step 2:

[1767] The server receives user data sent from the terminal and stores it in the database. The stored data includes the user's budget, family size, and excluded foods. The server analyzes the data and converts it into the necessary data structure. For example, if a user enters "Exclude nuts," nuts are added to the excluded foods list. The input data is user information sent from the terminal, and the output is the information stored in the database.

[1768] Step 3:

[1769] The device collects facial expressions and voice tone when the user inputs information. Equipped with a camera and microphone, the device captures the user's facial expressions and voice data in real time. This data is sent to an emotion engine for analysis. The input is data on facial expressions and voice tone, while the output is analyzed emotion data.

[1770] Step 4:

[1771] The emotion engine analyzes the collected data and sends it to the server. For example, it uses "EmotionRecognition" for facial expression analysis and "VoiceEmotionRecognition" for voice analysis to determine if the user is tired, stressed, etc. This information is then sent to the server. The input data is the collected and analyzed emotion data, and the output is sent to the server.

[1772] Step 5:

[1773] The server generates an optimal menu based on the user's settings and emotional data. For example, if the server detects that the user is tired, it prioritizes suggesting easy-to-prepare dishes. The generated menu data is returned to the terminal and displayed to the user. The input is the user's settings and emotional data, and the output is the generated menu.

[1774] Step 6:

[1775] The server generates a list of necessary ingredients based on the proposed menu. For example, it creates a specific list such as, "For Day 1's chicken curry, you'll need chicken, onions, curry powder, spices, etc." The ingredient list is converted into a detailed data structure and managed. The input is the proposed menu, and the output is the generated ingredient list.

[1776] Step 7:

[1777] The server places an order using the e-commerce site's API based on the confirmed list of ingredients and schedules the delivery date and time. For example, it might set "Purchase chicken, onions, and curry powder on [Month] [Day], and deliver on [Month] [Day]." It calls the API, adds the ingredients to the online shopping cart, and sets the delivery date and time. The input data is the generated list of ingredients, and the output is the completed order information.

[1778] Step 8:

[1779] The server tracks expenses in real time based on purchase history and manages them to stay within budget. It periodically checks budget progress to prevent users from overspending and displays warning messages as needed. For example, it might display, "Current expenses: ¥15,000, remaining budget: ¥15,000." The input data is purchase history, and the output is budget tracking and warning messages.

[1780] Example of a prompt

[1781] User-entered data: Budget of 30,000 yen, family of 4, nuts excluded. Facial expressions and voice tone were analyzed to determine the user was tired. Based on this information, suggest a simple meal plan for the user: "Dinner on day 1: Chicken curry, Day 2: Stir-fried vegetables."

[1782] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1783] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1784] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1785] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1786] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1787] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1788] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1789] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1790] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1791] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1792] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1793] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1794] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1795] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1796] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1797] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1798] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1799] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1800] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1801] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1802] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1803] The following is further disclosed regarding the embodiments described above.

[1804] (Claim 1)

[1805] A means for users to input their monthly budget, number of people, and foods they want to exclude,

[1806] A means of saving the entered data,

[1807] A means of generating a menu that does not include excluded foods within the budget, based on stored data,

[1808] A means for generating a list of necessary ingredients based on the generated menu,

[1809] A method for adding items to a cart via an e-commerce site based on a list of ingredients and scheduling delivery on a specified date,

[1810] A means of tracking expenses to ensure that monthly spending stays within budget based on saved data,

[1811] A system that includes this.

[1812] (Claim 2)

[1813] The system according to claim 1, wherein the menu generation means selects a menu that also takes into account the seasonality and nutritional balance of the ingredients.

[1814] (Claim 3)

[1815] The system according to claim 1, further comprising means for providing an interface for a user to view a list of ingredients and to add, delete, or adjust the quantities.

[1816] "Example 1"

[1817] (Claim 1)

[1818] A means for users to input their monthly budget, number of people, and foods they want to exclude,

[1819] A means of saving the entered data,

[1820] A means of generating a menu that does not include excluded foods within the budget, based on stored data,

[1821] A means for generating a list of necessary ingredients based on the generated menu,

[1822] A method for adding items to a cart via an e-commerce site based on a list of ingredients and scheduling delivery on a specified date,

[1823] A means of tracking expenses to ensure that monthly spending stays within budget based on saved data,

[1824] A means of checking the format of input content in real time and sending data only if the format is deemed correct,

[1825] A means of generating low-cost alternative recipes based on cost tracking,

[1826] A system that includes this.

[1827] (Claim 2)

[1828] The system according to claim 1, wherein the menu generation means selects a menu that also takes into account the seasonality and nutritional balance of the ingredients.

[1829] (Claim 3)

[1830] The system according to claim 1, further comprising means for providing an interface for a user to view a list of ingredients and to add, delete, or adjust the quantities.

[1831] "Application Example 1"

[1832] (Claim 1)

[1833] A means for users to input their monthly budget, number of people, and foods they want to exclude,

[1834] A means of saving the entered data,

[1835] A means of generating a menu that does not include excluded foods within the budget, based on stored data,

[1836] A means for generating a list of necessary ingredients based on the generated menu,

[1837] A method for adding items to an online shopping cart based on a list of necessary ingredients and scheduling delivery on a specified date,

[1838] A means of tracking expenses to ensure that monthly spending stays within budget based on saved data,

[1839] A means of providing an interface that allows users to check the progress of their budget and receive warning messages,

[1840] A system that includes this.

[1841] (Claim 2)

[1842] The system according to claim 1, wherein the menu generation means also takes into account the seasonality and nutritional balance of ingredients, and further includes means for calling an external e-commerce site API for purchasing products.

[1843] (Claim 3)

[1844] The system according to claim 1, further comprising means for providing an interface for a user to view a list of ingredients and to add, delete, or adjust quantities, and means for inputting prompt sentences into a generating AI model to generate recipes and ingredient information.

[1845] "Example 2 of combining an emotion engine"

[1846] (Claim 1) 【18...

Claims

1. A means for users to input their monthly budget, number of people, and foods they want to exclude, A means of saving the entered data, A means of generating a menu that does not include excluded foods within the budget, based on stored data, A means for generating a list of necessary ingredients based on the generated menu, A method for adding items to a cart via an e-commerce site based on a list of ingredients and scheduling delivery on a specified date, A means of tracking expenses to ensure that monthly spending stays within budget based on saved data, A system that includes this.

2. The system according to claim 1, wherein the menu generation means selects a menu that also takes into account the seasonality and nutritional balance of the ingredients.

3. The system according to claim 1, further comprising means for providing an interface for a user to view a list of ingredients and to add, delete, or adjust the quantity.

Images