system

Abstract

Provide a system. A user terminal includes a means for acquiring flyer data; A means for obtaining user information; A means for the server to generate a menu based on flyer data and user information; A server generates a recipe based on the generated menu; a means for the server to create a shopping list based on the generated recipe; a means for displaying the created menu, recipes, and shopping list to the user in the terminal; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, effectively planning, shopping, and cooking meals amidst busy daily lives has become a major challenge. Furthermore, planning menus while taking into account allergies, preferences, and budgets while utilizing sales information is time-consuming and often stressful for users. To address this situation, there is a need for an automated system that efficiently proposes menus tailored to the user and creates shopping lists, thereby reducing the time and effort required for cooking. [Means for solving the problem]

[0005] The present invention solves the above problems by providing a system including a means for a user terminal to acquire flyer data, a means for acquiring user information, a means for a server to generate a menu based on the flyer data and the user information, a means for the server to generate recipes based on the generated menu, a means for the server to create a shopping list based on the generated recipes, and a means for the terminal to display the generated menu, recipes, and shopping list to the user. This system allows users to efficiently plan menus, easily obtain necessary ingredients, and significantly reduce the time and stress involved in cooking.

[0006] The following are definitions of important terms contained in the claims.

[0007] A "user terminal" is an electronic device that can acquire and display information through user operation.

[0008] "Flyer data" refers to digital data of advertising information provided by supermarkets and other retail stores, such as product names, prices, discount information, and images.

[0009] "User information" refers to information about personal attributes and settings such as user preferences, allergy information, and budget.

[0010] A "server" is a computer system that can receive, process, and transmit information over a network.

[0011] A "menu" is a meal plan proposal that is made up of a combination of selected ingredients and cooking methods.

[0012] A "recipe" is a detailed instruction manual that includes a list of ingredients needed to prepare a dish and cooking instructions.

[0013] A "shopping list" is a list of ingredients and products needed to make a specific menu or recipe.

[0014] "Display means" refers to devices or technologies for visually conveying information to a user, such as a display or screen. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] This invention is a system in which a server generates and proposes menus to users based on flyer data and user information acquired by the user terminal. This system helps users to shop efficiently and plan and implement appropriate menus and recipes even in their busy daily lives.

[0037] Program processing

[0038] This system is divided into the following main processing steps: Specific operations at each step are as follows:

[0039] 1. Obtain flyer data

[0040] A user uses a device to download the latest flyer data for a specified supermarket. For example, the user selects a flyer for Supermarket X. The device accesses Supermarket X's online service, obtains data such as product name, price, discount information, and images, and sends this data to the server.

[0041] 2. Obtaining user information

[0042] The server retrieves the user's personal profile information from the database. For example, it confirms that User A's preference is Japanese food, that he / she has a dairy allergy, and that his / her weekly food budget is 5,000 yen.

[0043] 3. Menu generation

[0044] The server generates a menu tailored to the user based on the flyer data and user information it has acquired. For example, it selects chicken thighs, cabbage, and tomatoes from the flyer data, extracts the user's favorite Japanese recipe from a home cooking database, and suggests "chicken steamed with cabbage."

[0045] 4. Recipe suggestions

[0046] The server generates a detailed recipe based on the menu, including a list of specific ingredients (e.g., 200g chicken thighs, 1 / 2 head of cabbage, 1 tomato, salt, and pepper) and cooking instructions (e.g., 1. Roughly shred the cabbage. 2. Cut the chicken thighs into bite-sized pieces. 3. Steam the cabbage and chicken thighs).

[0047] 5. Create a shopping list

[0048] The server creates a shopping list based on recipe information. For example, the list might include "400g chicken thighs, 1 head of cabbage, and 2 tomatoes." The server compares the list with the ingredients available at home and removes unnecessary items.

[0049] 6. User Display

[0050] The device displays the created menu, recipes, and shopping list to the user. The user can review this information and make corrections as necessary. For example, the user may decide that they do not need to buy cabbage because they already have it at home and correct the list accordingly.

[0051] Specific examples

[0052] For example, suppose a user wants to make a simple, nutritious meal for a weekday dinner. The user downloads a flyer for Supermarket X from their device and checks the specials it contains. They learn that chicken thighs, cabbage, and tomatoes are on sale, and use the system to suggest a meal. The server considers the user's preferences and allergy information and suggests "steamed chicken with cabbage." A detailed recipe is generated, and a shopping list is also created. The user can review this information on their device, make any necessary corrections, and then go shopping. In this way, users can efficiently plan their meals and ensure the shopping and cooking processes proceed smoothly.

[0053] The processing flow will be explained below.

[0054] Step 1:

[0055] The user makes a request on the terminal to download the latest flyer data of the specified supermarket.

[0056] Step 2:

[0057] The device accesses the supermarket's online service and retrieves the latest flyer data (product name, price, discount information, images, etc.).

[0058] Step 3:

[0059] The terminal transmits the acquired flyer data to the server.

[0060] Step 4:

[0061] The server retrieves the user's personal profile information (preferences, allergy information, budget, etc.) from a database.

[0062] Step 5:

[0063] The server generates a menu tailored to the user based on the acquired flyer data and user information.

[0064] Step 6:

[0065] The server retrieves recipes using the selected ingredients from a home cooking database and generates a detailed recipe, including a specific ingredient list and cooking instructions.

[0066] Step 7:

[0067] The server creates a shopping list of the necessary ingredients based on the generated recipe.

[0068] Step 8:

[0069] The server sends the shopping list to the terminal.

[0070] Step 9:

[0071] The device displays the created menu, recipes, and shopping list to the user, who can review and modify them as necessary.

[0072] Step 10:

[0073] The user goes shopping at the supermarket based on the shopping list and cooks using the purchased ingredients according to the recipe.

[0074] Example 1

[0075] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0076] Modern users face the challenge of efficiently shopping and planning and implementing appropriate menus and recipes in their busy daily lives. Conventional methods require users to collect market information and manually plan menus tailored to their individual needs, which takes a lot of time and effort. For this reason, there is a demand for a system that allows users to easily plan and implement nutritionally balanced meals while efficiently purchasing ingredients.

[0077] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0078] In this invention, the server includes means for generating a meal plan based on market information and user information, means for generating cooking instructions based on the generated meal plan, and means for creating a shopping list based on the generated cooking instructions. This allows users to automatically receive market information and meal plans and recipes tailored to their individual needs, allowing them to shop efficiently and easily plan and carry out appropriate meals.

[0079] A "user terminal" is an electronic device that allows a user to input and receive information.

[0080] "Market information" is data on products sold in specific markets or stores, and includes product names, prices, discount information, images, and the like.

[0081] "User information" is data about an individual user, including preferences, allergy information, budget, and the like.

[0082] A "meal plan" is a menu created based on the user's market information and user information.

[0083] "Cooking Instructions" are detailed instructions and ingredient lists for dishes based on a meal plan.

[0084] A "shopping list" is a list of ingredients and items needed based on the cooking procedure.

[0085] A "system" is a collection of hardware and software, including user terminals and servers, for realizing specific functions.

[0086] A "server" is a centralized computer system that processes and stores data.

[0087] This invention is a system that generates meal plans based on market information and user information and proposes them to users. This system helps users shop efficiently and plan and implement appropriate menus and recipes.

[0088] The system consists of user devices, servers, and supporting hardware and software. Specifically, it includes user devices such as smartphones and tablets, a server with a database, and an internet connection. The software supporting this hardware includes a market information acquisition API, a user profile management system, data analysis algorithms, machine learning models, and a user interface application.

[0089] Users can operate the application using a smartphone or tablet to obtain the latest information on a specific market. Market information includes product names, prices, discount information, images, etc. This information is sent to the server through a market information acquisition API. The server then retrieves personal profile information such as the user's preferences, allergies, and budget from a database.

[0090] The server generates an optimal meal plan for the user based on the acquired market information and user information. Data analysis algorithms and machine learning models are used to generate the meal plan. Specifically, specific ingredients are selected from market information, and appropriate recipes are extracted based on the user's preferences and allergy information. Through this series of processes, a specific recipe, such as "chicken steamed with cabbage," is suggested.

[0091] The server then creates a shopping list based on the generated recipes and ingredients list, which is then compared with the existing household inventory to eliminate unnecessary items, enabling efficient shopping.

[0092] The user terminal displays the meal plan, cooking instructions, and shopping list received from the server on a user interface. The user can review the information and make corrections as needed. For example, if there is already cabbage at home, the user can remove it from the shopping list.

[0093] A concrete example of this system implementation is when a user wants to make a simple and nutritious meal for a weekday dinner. The user retrieves flyer data for a specific market from their device, checks discounted products, and enters them into the system. The server takes into account the user's preferences and allergy information and suggests, for example, "chicken steamed with cabbage." A detailed recipe and shopping list are generated and displayed on the user interface.

[0094] As an example of a prompt sentence, the following sentence could be fed into a generative AI model:

[0095] "Please explain the system in which a user retrieves flyer data for a specific market from a device, and a server generates and proposes a meal plan to the user based on the user's preferences and allergy information. Specifically, please describe in detail each step from retrieving flyer data to proposing recipes and creating a shopping list."

[0096] In this way, by utilizing generative AI models, it is possible to provide users with efficient and nutritionally balanced meal plans.

[0097] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0098] Step 1:

[0099] The user uses a device to obtain market information. Specifically, the user selects a market, and the device obtains the latest market data through the market information acquisition API. The input here is the market information selected by the user, and the output is market data such as product name, price, discount information, and images. The device temporarily stores this data and sends it to the server.

[0100] Step 2:

[0101] The server retrieves user information. Based on the user ID entered when the user logged in to the application, the server uses an SQL query to retrieve the user's personal profile information from the database. The input is the user ID, and the output is the user's personal profile information, such as preferences, allergies, and budget. This information is stored in the user object within the program.

[0102] Step 3:

[0103] The server generates a meal plan based on market information and user information. The server uses data analysis algorithms and machine learning models to analyze the input market data and user information. The input is market data and user information, and the output is a meal plan suitable for the user (e.g., "chicken steamed with cabbage"). The server extracts related recipes from a home cooking database to suggest more detailed recipes.

[0104] Step 4:

[0105] The server generates detailed cooking instructions based on the meal plan. Specifically, it writes out the ingredient list and cooking instructions from the database and stores the data in a packet in JSON format. The input is the meal plan content, and the output is the ingredient list and detailed cooking instructions (e.g., "200g chicken thighs, 1 / 2 head of cabbage, 1 tomato, salt, pepper" and specific cooking instructions such as "Roughly chop the cabbage. Cut the chicken thighs into bite-sized pieces. Steam the cabbage and chicken thighs.").

[0106] Step 5:

[0107] The server creates a shopping list based on the cooking instructions. The input is the cooking instructions and the household inventory list, and the output is a shopping list with unnecessary items removed. The server compares the existing household inventory list with the newly created ingredient list and removes unnecessary items (e.g., if cabbage is already in the household, remove it from the shopping list).

[0108] Step 6:

[0109] The device displays the created meal plan, cooking instructions, and shopping list to the user. The input is the data received from the server, and the output is the information displayed on the user interface. The device application retrieves this latest data and acts as an interface for the user to review and modify. The user can then review the displayed information, modify it as necessary, and go shopping.

[0110] (Application example 1)

[0111] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0112] Conventional shopping planning support systems have the problem that it takes a lot of time and effort for users to efficiently shop and plan and implement appropriate menus and recipes in their busy daily lives. In particular, the complicated process of generating menus based on the latest flyer information and then ordering the ingredients through a delivery service is a challenge. In addition, the system's inability to provide optimal menu suggestions that meet the user's specific needs can lead to low user satisfaction.

[0113] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0114] In this invention, the server includes means for generating a menu based on flyer data and user information, means for generating recipes based on the generated menu, means for creating a shopping list based on the generated recipe, and means for the terminal to generate prompt sentences using a generative AI model to suggest an optimal menu in response to user requests, and means for providing a delivery service based on the generated shopping list. This enables users to efficiently plan their shopping, smoothly obtain the necessary ingredients through a delivery service, and implement an optimal menu.

[0115] A "user terminal" is a communication device operated by a user, and includes a smartphone, a tablet terminal, a personal computer, and the like.

[0116] "Flyer data" is product information provided by a sales store, and includes data such as product name, price, discount information, and images.

[0117] "User information" is information that indicates the individual characteristics and preferences of a user, including food preferences, allergy information, budget, and the like.

[0118] A "server" is a computer system that processes large amounts of data, and receives and processes data from user terminals.

[0119] The "menu creation means" has the function of creating a specific meal plan based on the acquired flyer data and user information.

[0120] The "recipe generation means" has a function of creating specific cooking procedures and a list of ingredients based on the generated menu.

[0121] The "shopping list creation means" has a function of creating a list of ingredients needed based on the generated recipe.

[0122] The "delivery service providing means" realizes the function of ordering and delivering the necessary ingredients based on the generated shopping list.

[0123] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to suggest optimal menus based on user requests.

[0124] A "prompt sentence" is a sentence input into a generative AI model, and is text that describes the user's requests and conditions.

[0125] The present invention provides a system that combines a user terminal, a server, and a delivery service to efficiently plan shopping and propose menus. Specific embodiments of the system will be described in detail below.

[0126] 1. Hardware Configuration

[0127] In this embodiment, the following hardware is used.

[0128] User devices: smartphones, tablets, personal computers, etc.

[0129] Server: A computer system that processes large amounts of data

[0130] Delivery service: online ordering and delivery system

[0131] 2. Software Configuration

[0132] The system uses the following software:

[0133] Python: Used to develop programs

[0134] Flask: a web framework

[0135] The Requests library: Executing web requests

[0136] SQLite or Firebase: a database for storing user information

[0137] Generative AI model: An artificial intelligence model for proposing optimal menus based on user requests

[0138] 3. Data processing and calculations

[0139] The main data processing and calculation flow performed by the system is shown below.

[0140] (1) Obtaining flyer data

[0141] Users can access the latest flyer data for a designated store from their smartphone or other device. This is done by accessing the store's online service and downloading data such as product name, price, discount information, and images. This information is then sent to a server via the user's device.

[0142] (2) Acquisition of user information

[0143] The server retrieves the user's profile information from a database, including the user's food preferences, allergy information, budget, etc.

[0144] (3) Menu generation

[0145] The server generates the optimal menu based on the acquired flyer data and user information. For example, it selects special offers on chicken thighs, cabbage, tomatoes, etc. from the flyer data and suggests "steamed chicken with cabbage" taking into account the user's preferences and allergies.

[0146] (4) Recipe suggestions

[0147] Once the menu is decided, the server generates a detailed recipe, including a list of specific ingredients (e.g., 200g chicken thighs, 1 / 2 head of cabbage, 1 tomato, salt, and pepper) and cooking instructions (e.g., 1. Roughly shred the cabbage. 2. Cut the chicken thighs into bite-sized pieces. 3. Steam the cabbage and chicken thighs).

[0148] (5) Creating a shopping list

[0149] The server creates a shopping list based on the generated recipe. For example, the list might include "400g of chicken thighs, 1 head of cabbage, and 2 tomatoes." It then compares the list with ingredients already in the home and eliminates any duplicates.

[0150] (6) Providing delivery services

[0151] Implement a means to order the necessary ingredients from a delivery service based on the generated shopping list. This includes the process of ordering ingredients from a user terminal through a delivery service and having them delivered to a specified location.

[0152] 4. Use of generative AI models

[0153] The server uses a generative AI model to propose the optimal menu based on the user's request. Specifically, the AI ​​generates an appropriate menu based on the prompt text entered by the user.

[0154] Prompt Sentence Examples

[0155] "Please suggest a balanced dinner menu based on the user's preferences and the latest flyer information. It should be Japanese, dairy-free, and within a budget of 5,000 yen."

[0156] In this way, the present invention enables users to efficiently plan their shopping, smoothly obtain the necessary ingredients through a delivery service, and implement an optimal menu.

[0157] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0158] Step 1:

[0159] The user terminal acquires the latest flyer data of the designated store.

[0160] Input: User-specified store identification information.

[0161] Operation: The user's device accesses the online service and downloads flyer data such as product names, prices, discount information, and images.

[0162] Output: The retrieved flyer data.

[0163] Step 2:

[0164] The server retrieves the user information from the database.

[0165] Input: User ID.

[0166] How it works: The server accesses a local or cloud database (e.g., SQLite, Firebase) to retrieve information about the user, such as their food preferences, allergies, and budget.

[0167] Output: User information.

[0168] Step 3:

[0169] The server generates the optimal menu based on flyer data and user information.

[0170] Input: Flyer data, user information.

[0171] Operation: The server selects special sale information from flyer data and creates a menu taking into account the user's preferences and allergy information.

[0172] Output: The generated menu.

[0173] Step 4:

[0174] The server generates detailed recipes based on the generated menu.

[0175] Input: The generated menu.

[0176] How it works: The server extracts suitable recipes from a home cooking database and generates specific ingredient lists and cooking instructions.

[0177] Output: A detailed recipe.

[0178] Step 5:

[0179] The server creates a shopping list based on the generated recipe.

[0180] Input: A detailed recipe.

[0181] How it works: The server creates a list of ingredients needed based on the recipe, compares it with the list of ingredients already in the house, and weeds out any duplicates.

[0182] Output: Shopping list.

[0183] Step 6:

[0184] The server provides delivery services based on the shopping list.

[0185] Enter: shopping list.

[0186] How it works: The server uses the delivery service's API to order ingredients and arrange for delivery to the specified location.

[0187] Output: Delivery order confirmation information.

[0188] Step 7:

[0189] The device uses a generative AI model to generate prompts that suggest the optimal menu based on the user's requests.

[0190] Input: User request (e.g., "Please suggest a balanced dinner menu based on the user's preferences and the latest flyer information. Japanese food, dairy-free, within a budget of 5,000 yen.").

[0191] How it works: Enter a prompt into the generative AI model, and the AI ​​will generate an appropriate menu.

[0192] Output: The optimal menu suggested to the user.

[0193] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0194] This invention is a menu suggestion and recipe provision system that combines an emotion engine that recognizes the user's emotions. This system makes the user's eating habits more efficient by having the server generate optimal menus, recipes, and shopping lists based on flyer data, user information, and emotion information acquired by the user's terminal.

[0195] Program processing

[0196] This system is divided into the following main processing steps: Specific operations at each step are as follows:

[0197] 1. Obtain flyer data

[0198] The user requests the download of the latest flyer data for the specified supermarket on their device. The device accesses the supermarket's online service and obtains data such as product names, prices, discount information, and images. The obtained flyer data is then sent to the server.

[0199] 2. Obtaining user information

[0200] The server retrieves the user's personal profile information (preferences, allergy information, budget, etc.) from the database. For example, it confirms that User A's preference is Japanese food, that he has a dairy allergy, and that his weekly food budget is 5,000 yen.

[0201] 3. Acquiring emotional information

[0202] Using an emotion engine built into or connected to the user's device, the system acquires emotional information from the user's facial expressions and tone of voice while they are browsing the flyer. For example, it can recognize emotions such as "tired" or "stressed."

[0203] 4. Menu generation

[0204] The server generates a menu suited to the user based on the acquired flyer data, user information, and emotional information. For example, if the user is tired, it will prioritize simple recipes with short cooking times. It will also select chicken thighs, cabbage, and tomatoes from the flyer data and suggest "chicken steamed with cabbage."

[0205] 5. Recipe suggestions

[0206] The server generates a detailed recipe based on the generated menu, including a list of specific ingredients (e.g., 200g chicken thighs, 1 / 2 head of cabbage, 1 tomato, salt, and pepper) and cooking instructions (e.g., 1. Roughly shred the cabbage. 2. Cut the chicken thighs into bite-sized pieces. 3. Steam the cabbage and chicken thighs).

[0207] 6. Create a shopping list

[0208] The server creates a shopping list based on recipe information. For example, the list might include "400g chicken thighs, 1 head of cabbage, and 2 tomatoes." The server compares the list with the ingredients available at home and removes unnecessary items.

[0209] 7. User Display

[0210] The device displays the meal plan, recipes, and shopping list to the user, who can review and modify them as needed. For example, the device may adjust the display to present particularly simple steps for a tired user.

[0211] Specific examples

[0212] For example, suppose a user finds a sale flyer at a supermarket on a weekday evening and needs to prepare dinner. The user downloads the supermarket flyer data using their device and checks the sale items included in the flyer. The emotion engine recognizes the emotion "tired" from the user's facial expression and tone of voice. The server takes into account the user's preferences and allergy information, as well as the user's emotional fatigue information, and suggests "steamed chicken with cabbage," which is easy to prepare and has excellent nutritional value, as a menu item. A detailed recipe and shopping list are generated and displayed on the user's device. The user can review the list, make any necessary edits, then go shopping at the supermarket and easily prepare the dish at home.

[0213] In this way, the present invention reflects the user's emotional state and provides appropriate menus and recipes, thereby reducing the time and effort required for cooking and easing the burden on the user.

[0214] The processing flow will be explained below.

[0215] Step 1:

[0216] The user makes a request on the terminal to download the latest flyer data of the specified supermarket.

[0217] Step 2:

[0218] The device accesses the online service of the specified supermarket and obtains data such as product name, price, discount information, images, etc. The obtained flyer data is then sent to the server.

[0219] Step 3:

[0220] The server retrieves the user's personal profile information (preferences, allergy information, budget, etc.) from a database.

[0221] Step 4:

[0222] Using an emotion engine built into or connected to the user's device, the system acquires emotional information from the user's facial expressions and tone of voice while they are browsing the flyer. For example, it can recognize emotions such as "tired" or "stressed."

[0223] Step 5:

[0224] The emotion information acquired by the user terminal is transmitted to the server.

[0225] Step 6:

[0226] The server generates a menu suited to the user based on the acquired flyer data, user information, and emotional information. For example, if the user is tired, it will prioritize simple recipes with short cooking times.

[0227] Step 7:

[0228] The server retrieves recipes using the selected ingredients from a home cooking database and generates a detailed recipe, including a specific ingredient list and cooking instructions.

[0229] Step 8:

[0230] The server creates a shopping list based on the generated recipe. For example, the list might include "400g of chicken thighs, 1 head of cabbage, and 2 tomatoes." The server compares the list with the ingredients available at home and removes unnecessary items.

[0231] Step 9:

[0232] The server sends the shopping list to the terminal.

[0233] Step 10:

[0234] The device displays the created menu, recipes, and shopping list to the user, who can review them and make any necessary corrections.

[0235] Step 11:

[0236] The user goes shopping at the supermarket based on the shopping list and cooks food according to the recipe using the purchased ingredients.

[0237] Example 2

[0238] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0239] In recent years, for many users who lead busy lives, preparing meals every day requires time and effort. In particular, it is difficult to propose menus that take into account the user's emotional state and preferences. Therefore, there is a need for menu suggestions that are appropriate for users who are tired or have specific preferences or allergies. There is also a need for support in creating efficient shopping lists and cooking based on those lists. There is a need for systems and methods that can solve these problems.

[0240] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for generating a menu based on flyer data, user information, and emotion information, a means for generating recipes based on the generated menu, and a means for creating a shopping list based on the generated recipe. This makes it possible to provide an optimal menu according to the user's emotions and individual needs, and to save the effort of creating a shopping list and cooking.

[0241] "User terminal" refers to an electronic device such as a computer device or smartphone that is directly operated by a user.

[0242] "Flyer data" is data containing information on the latest products offered by retailers such as supermarkets, and includes product names, prices, discount information, images, and the like.

[0243] "User information" refers to information about a user's personal profile, including preferences, allergy information, budget, and the like.

[0244] "Emotional information" refers to data that indicates the emotional state of the user, such as facial expressions and tone of voice.

[0245] "Menu" refers to the combination of dishes and menus suggested to optimize the user's diet.

[0246] "Recipe" refers to a document or information that includes a specific list of ingredients and cooking instructions based on a menu.

[0247] A "shopping list" is a list of ingredients needed to prepare a recipe, and is intended to support efficient shopping.

[0248] A "server" refers to a computer system that processes requests from terminals via a network and sends and receives necessary data.

[0249] This invention is a menu suggestion and recipe provision system that combines an emotion engine that recognizes the user's emotions. Based on flyer data, user information, and emotion information acquired by the user's terminal, the server generates optimal menus, recipes, and shopping lists and displays them to the user, thereby streamlining the user's eating habits.

[0250] Hardware and software used

[0251] User device: An electronic device that can be directly operated by a user, such as a computer or smartphone, to which a camera, microphone, etc. are connected.

[0252] Server: A computer system that processes requests from terminals and sends and receives data over a network.

[0253] Emotion engine: Software that recognizes emotions from the user's facial expressions and tone of voice. A specific example is the use of machine learning libraries such as TENSORFLOW (registered trademark).

[0254] Database: A database system such as MySQL (registered trademark) for storing user information and flyer data.

[0255] Programming languages ​​and libraries: Python, Pandas, HTTP requests, etc.

[0256] Specific operation explanation

[0257] Obtaining flyer data

[0258] When a user downloads flyer information for a particular supermarket on their device, the device accesses the online service using an HTTP request to obtain data such as product names, prices, discount information, images, etc. This data is temporarily stored in local storage and then sent to the server.

[0259] Obtaining user information

[0260] The server queries the database based on the user ID and obtains information such as the user's preferences, allergies, budget, etc. The data is loaded into memory and handed over to the next process.

[0261] Acquiring emotional information

[0262] The user device uses a camera and microphone to collect facial expressions and tone of voice, which are then analyzed by the emotion engine. TensorFlow is used for emotion recognition, and the estimated results are sent to the server.

[0263] Menu and recipe generation

[0264] The server runs an algorithm to generate optimal menus based on collected flyer data, user information, and emotional information. For example, if the server detects that the user is tired, it prioritizes recipes with short cooking times. This is done using Python scripts and the Pandas library. From the generated menu, a detailed recipe is created, including a specific ingredient list and cooking instructions.

[0265] Creating a shopping list

[0266] Based on the generated recipe, the necessary ingredients are compiled into a shopping list, which is then compared with the ingredients in your home and unnecessary items are removed to create the final shopping list.

[0267] What the user sees

[0268] Finally, the generated menu, detailed recipes, and shopping list are displayed on the user's device, where the user can review the information provided and make any necessary corrections.

[0269] Prompt Sentence Examples

[0270] For example, an example of a prompt sentence to be input into a generative AI model is, "When a user finds a sale flyer at the supermarket on a weekday evening and needs to prepare dinner, please explain the process of the system recognizing the user's tiredness from their facial expression and tone of voice, and suggesting a menu that is easy to prepare while taking those emotions into consideration."

[0271] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0272] Step 1: Obtain flyer data

[0273] User: The user operates the device and issues a request to download flyer data for a specific supermarket.

[0274] Input: Supermarket flyer URL

[0275] Terminal: Upon receiving a request from the user, the terminal uses an HTTP request to access the supermarket's online service and retrieve flyer data.

[0276] Data processing: The acquired data includes product names, prices, discount information, images, etc. This data is parsed, organized, and saved.

[0277] Output: Organized flyer data

[0278] Device: The organized flyer data is temporarily stored in local storage, and then sent to the server via an HTTP POST request.

[0279] Step 2: Get user information

[0280] Server: Queries the database based on the user ID and retrieves the user's personal profile information.

[0281] Input: User ID

[0282] Data processing: Information retrieved from the database includes user preferences, allergies, budget, etc. This information is loaded into memory and organized for further processing.

[0283] Output: User's personal profile information

[0284] Server: Records the acquired user information in a log and passes it to the subsequent menu generation process.

[0285] Step 3: Acquiring emotional information

[0286] Device: Uses a camera and microphone to capture your facial expressions and tone of voice.

[0287] Input: User facial and voice data

[0288] Data processing: Using an emotion recognition engine (e.g., TensorFlow), the collected data is analyzed to estimate the user's emotional state, and emotional information such as fatigue and stress levels is obtained.

[0289] Output: User's emotional information

[0290] Device: Sends analyzed emotion information to the server.

[0291] Step 4: Create a menu

[0292] Server: Runs an algorithm to generate a menu suitable for the user based on the acquired flyer data, user information, and emotional information.

[0293] Input: flyer data, user information, emotion information

[0294] Data processing: Using algorithms, recipe information stored in the database is filtered based on evaluation criteria (ease of use, nutritional balance, cooking time). Based on emotional information, for example, recipes with short cooking times are prioritized for tired users.

[0295] Output: Optimal meal plan

[0296] Server: Pass the generated menu plan to the next recipe generation step.

[0297] Step 5: Recipe suggestions

[0298] Server: Runs a script that generates detailed recipes based on optimal menu plans.

[0299] Input: Optimal meal plan

[0300] Data processing: Using the Pandas library, generate a specific list of ingredients (e.g., 200g chicken thighs, 1 / 2 head of cabbage, 1 tomato, salt, and pepper) and cooking instructions (e.g., roughly chop the cabbage. Cut the chicken thighs into bite-sized pieces. Steam the cabbage and chicken thighs).

[0301] Output: Detailed recipe

[0302] Server: Pass the generated recipe to the shopping list generation step.

[0303] Step 6: Create a shopping list

[0304] Server: Based on the recipe information, it creates a list of the necessary ingredients and creates a shopping list.

[0305] Input: Detailed recipe

[0306] Data processing: Match the data with the grocery list at home, filter out unnecessary items, and generate the final shopping list.

[0307] Output: Final shopping list

[0308] Server: Sends the generated shopping list to the device.

[0309] Step 7: Display to the user

[0310] Terminal: Displays the menu, recipes, and shopping list received from the server on the user interface.

[0311] Input: Menus, recipes, shopping lists

[0312] Data processing: Organize and display information in a format that is easy for users to view. If the user makes any corrections, the data is sent back to the server.

[0313] Output: Menu, recipes, and shopping list displayed to the user

[0314] User: Review the information provided and make corrections as needed.

[0315] (Application example 2)

[0316] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0317] In modern society, as user lifestyles become more diverse, systems that suggest optimal meals based on an individual's emotional and health states are becoming increasingly important. However, conventional systems often suggest menus based solely on the user's preferences and budget, without taking the user's emotional state into consideration, resulting in low user satisfaction and low utilization rates. Furthermore, for busy modern people, finding the optimal meal for their emotional and physical states can be a significant burden. Therefore, there is a need for a system that utilizes the user's emotional information to suggest optimal menus.

[0318] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for acquiring flyer data, a means for acquiring emotion information, a means for generating a menu based on user information, a means for generating recipes based on the generated menu, and a means for creating a shopping list based on the generated recipe. This makes it possible to propose an optimal menu that reflects the user's emotional state.

[0319] A "user terminal" is an electronic device used by a user, and includes smartphones, tablets, PCs, etc.

[0320] "Flyer data" is advertising material issued by supermarkets and stores, and includes product names, prices, discount information, images, and the like.

[0321] "Emotional information" is data on the user's emotional state based on facial expressions, tone of voice, and other biological information.

[0322] "User information" refers to personal profile information such as the user's preferences, allergy information, budget, etc.

[0323] A "server" is a central system that processes and stores information on a network and communicates with clients (user terminals).

[0324] A "menu" is a meal plan made up of a combination of multiple dishes and ingredients.

[0325] A "recipe" is a list of ingredients and specific steps for making a dish.

[0326] A "shopping list" is a list of ingredients and products needed based on a recipe.

[0327] A "system" is a collection of devices and programs in which multiple functional components work in conjunction with one another.

[0328] This invention is a system that utilizes user emotion information to propose optimal menus for food delivery. This system includes a user terminal, an emotion engine, a server, and a network for communication among these.

[0329] System configuration

[0330] The user terminal is an electronic device such as a smartphone, tablet, or PC through which the user accesses the system. The user terminal has an internet connection function and allows input of the emotion engine and user information.

[0331] The emotion engine is software or hardware that analyzes a user's facial expressions, tone of voice, and other biometric information to generate emotional information about the user. The emotion engine is either built into the user terminal or operates as an external device connected to the user terminal.

[0332] The server is a central system that processes and stores emotion information, flyer data, and user information over the network. The server manages data such as user profiles, flyer databases, and recipe databases, and generates optimal menus and recipes for users.

[0333] System Operation

[0334] The operation of the system is as follows.

[0335] 1. Obtain flyer data:

[0336] The user's device downloads the latest menu information for the specified food delivery service and transmits it to the server, including product names, prices, images, etc.

[0337] 2. Acquiring emotional information:

[0338] The user device uses an emotion engine to obtain emotional information (e.g., "I'm tired" or "I'm stressed") from the user's facial expressions and tone of voice, and sends it to the server.

[0339] 3. Obtaining user information:

[0340] The server retrieves personal profile information such as the user's preferences, allergy information, budget, etc. from the database. For example, it confirms that the user's preference is Japanese food, that they have a dairy allergy, and that their weekly food budget is 5,000 yen.

[0341] 4. Menu generation:

[0342] The server generates a menu tailored to the user based on the flyer data, emotional information, and user information it has acquired. For example, if the user is tired, it will prioritize and suggest simple menus that require less time to cook.

[0343] 5. Recipe generation and shopping list creation:

[0344] Based on the generated menu, the server generates a detailed recipe and a shopping list for its execution, including a specific list of ingredients and cooking instructions.

[0345] 6. User Visibility:

[0346] The user terminal displays the menu, recipes, and shopping list sent from the server, and the user can check them and place orders as necessary.

[0347] Specific examples

[0348] Suppose a user is browsing a food delivery menu on a weekday evening. The user's device downloads the latest menu information (product name, price, and image), and the emotion engine recognizes the user's emotion, "tired," from their facial expression and tone of voice. The server takes this emotional information, along with the user's preferences and allergies, into consideration to suggest a menu that is easy to prepare and nutritious. For example, it suggests a dish called "chicken steamed with cabbage," and a detailed recipe and list of ingredients are displayed on the user's device.

[0349] Prompt Sentence Examples

[0350] "The user is extremely tired. Please suggest a menu that you would recommend at this time."

[0351] "Emotional information: I'm tired. Please suggest a menu that requires little cooking time and is within a budget of 5,000 yen."

[0352] In this way, it becomes possible to propose an optimal menu that reflects the user's emotional state.

[0353] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0354] Step 1:

[0355] The user device obtains the latest menu information from a specified food delivery service. Specifically, the user device accesses the food delivery service's API via the Internet and downloads menu information (product name, price, image, etc.). This menu information is sent from the user device to a server for use in subsequent processing steps. The input is the menu information from the food delivery service API, and the output is the menu information sent to the server.

[0356] Step 2:

[0357] The user device uses an emotion engine to obtain the user's emotional information. Specifically, the emotion engine analyzes the user's facial expression and tone of voice in real time and generates emotional information (for example, "tired" or "stressed"). This emotional information is sent from the user device to the server. The input is the user's facial expression and tone of voice, and the output is the emotional information sent to the server.

[0358] Step 3:

[0359] The server retrieves the user's personal profile information from a database, including preferences, allergies, budget, etc. This information is then combined with other data to generate an optimal menu. The input is a query to the server's database, and the output is the retrieved user information.

[0360] Step 4:

[0361] The server generates a menu suitable for the user based on the acquired flyer data, emotional information, and user information. Specifically, it prioritizes menus with short cooking times and menus with a relaxing effect based on emotional information (e.g., "tired"). The inputs are flyer data, emotional information, and user information, and the output is the generated menu.

[0362] Step 5:

[0363] The server generates a detailed recipe and a shopping list based on the generated menu. Specifically, the recipe includes a list of specific ingredients (e.g., 200g chicken thighs, 1 / 2 head of cabbage, 1 tomato) and cooking instructions (e.g., 1. Roughly chop the cabbage. 2. Cut the chicken thighs into bite-sized pieces), and the shopping list lists the ingredients needed based on the recipe. The input is the generated menu, and the output is a detailed recipe and a shopping list.

[0364] Step 6:

[0365] The user terminal displays the menu, recipes, and shopping list sent from the server. Specifically, the menu, recipes, and shopping list are displayed on the screen of the user terminal, and the user checks them and places an order as necessary. The system also encourages the user to select a menu based on their emotions when ordering. The input is the data sent from the server, and the output is the menu, recipes, and shopping list displayed on the user terminal.

[0366] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0367] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0368] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0369] [Second embodiment]

[0370] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0371] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0372] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0373] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0374] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0375] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0376] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0377] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0378] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0379] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0380] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0381] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0382] This invention is a system in which a server generates and proposes menus to users based on flyer data and user information acquired by the user terminal. This system helps users to shop efficiently and plan and implement appropriate menus and recipes even in their busy daily lives.

[0383] Program processing

[0384] This system is divided into the following main processing steps: Specific operations at each step are as follows:

[0385] 1. Obtain flyer data

[0386] A user uses a device to download the latest flyer data for a specified supermarket. For example, the user selects a flyer for Supermarket X. The device accesses Supermarket X's online service, obtains data such as product name, price, discount information, and images, and sends this data to the server.

[0387] 2. Obtaining user information

[0388] The server retrieves the user's personal profile information from the database. For example, it confirms that User A's preference is Japanese food, that he / she has a dairy allergy, and that his / her weekly food budget is 5,000 yen.

[0389] 3. Menu generation

[0390] The server generates a menu tailored to the user based on the flyer data and user information it has acquired. For example, it selects chicken thighs, cabbage, and tomatoes from the flyer data, extracts the user's favorite Japanese recipe from a home cooking database, and suggests "chicken steamed with cabbage."

[0391] 4. Recipe suggestions

[0392] The server generates a detailed recipe based on the menu, including a list of specific ingredients (e.g., 200g chicken thighs, 1 / 2 head of cabbage, 1 tomato, salt, and pepper) and cooking instructions (e.g., 1. Roughly shred the cabbage. 2. Cut the chicken thighs into bite-sized pieces. 3. Steam the cabbage and chicken thighs).

[0393] 5. Create a shopping list

[0394] The server creates a shopping list based on recipe information. For example, the list might include "400g chicken thighs, 1 head of cabbage, and 2 tomatoes." The server compares the list with the ingredients available at home and removes unnecessary items.

[0395] 6. User Display

[0396] The device displays the created menu, recipes, and shopping list to the user. The user can review this information and make corrections as necessary. For example, the user may decide that they do not need to buy cabbage because they already have it at home and correct the list accordingly.

[0397] Specific examples

[0398] For example, suppose a user wants to make a simple, nutritious meal for a weekday dinner. The user downloads a flyer for Supermarket X from their device and checks the specials it contains. They learn that chicken thighs, cabbage, and tomatoes are on sale, and use the system to suggest a meal. The server considers the user's preferences and allergy information and suggests "steamed chicken with cabbage." A detailed recipe is generated, and a shopping list is also created. The user can review this information on their device, make any necessary corrections, and then go shopping. In this way, users can efficiently plan their meals and ensure the shopping and cooking processes proceed smoothly.

[0399] The processing flow will be explained below.

[0400] Step 1:

[0401] The user makes a request on the terminal to download the latest flyer data of the specified supermarket.

[0402] Step 2:

[0403] The device accesses the supermarket's online service and retrieves the latest flyer data (product name, price, discount information, images, etc.).

[0404] Step 3:

[0405] The terminal transmits the acquired flyer data to the server.

[0406] Step 4:

[0407] The server retrieves the user's personal profile information (preferences, allergy information, budget, etc.) from a database.

[0408] Step 5:

[0409] The server generates a menu tailored to the user based on the acquired flyer data and user information.

[0410] Step 6:

[0411] The server retrieves recipes using the selected ingredients from a home cooking database and generates a detailed recipe, including a specific ingredient list and cooking instructions.

[0412] Step 7:

[0413] The server creates a shopping list of the necessary ingredients based on the generated recipe.

[0414] Step 8:

[0415] The server sends the shopping list to the terminal.

[0416] Step 9:

[0417] The device displays the created menu, recipes, and shopping list to the user, who can review and modify them as necessary.

[0418] Step 10:

[0419] The user goes shopping at the supermarket based on the shopping list and cooks using the purchased ingredients according to the recipe.

[0420] Example 1

[0421] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0422] Modern users face the challenge of efficiently shopping and planning and implementing appropriate menus and recipes in their busy daily lives. Conventional methods require users to collect market information and manually plan menus tailored to their individual needs, which takes a lot of time and effort. For this reason, there is a demand for a system that allows users to easily plan and implement nutritionally balanced meals while efficiently purchasing ingredients.

[0423] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0424] In this invention, the server includes means for generating a meal plan based on market information and user information, means for generating cooking instructions based on the generated meal plan, and means for creating a shopping list based on the generated cooking instructions. This allows users to automatically receive market information and meal plans and recipes tailored to their individual needs, allowing them to shop efficiently and easily plan and carry out appropriate meals.

[0425] A "user terminal" is an electronic device that allows a user to input and receive information.

[0426] "Market information" is data on products sold in specific markets or stores, and includes product names, prices, discount information, images, and the like.

[0427] "User information" is data about an individual user, including preferences, allergy information, budget, and the like.

[0428] A "meal plan" is a menu created based on the user's market information and user information.

[0429] "Cooking Instructions" are detailed instructions and ingredient lists for dishes based on a meal plan.

[0430] A "shopping list" is a list of ingredients and items needed based on the cooking procedure.

[0431] A "system" is a collection of hardware and software, including user terminals and servers, for realizing specific functions.

[0432] A "server" is a centralized computer system that processes and stores data.

[0433] This invention is a system that generates meal plans based on market information and user information and proposes them to users. This system helps users shop efficiently and plan and implement appropriate menus and recipes.

[0434] The system consists of user devices, servers, and supporting hardware and software. Specifically, it includes user devices such as smartphones and tablets, a server with a database, and an internet connection. The software supporting this hardware includes a market information acquisition API, a user profile management system, data analysis algorithms, machine learning models, and a user interface application.

[0435] Users can operate the application using a smartphone or tablet to obtain the latest information on a specific market. Market information includes product names, prices, discount information, images, etc. This information is sent to the server through a market information acquisition API. The server then retrieves personal profile information such as the user's preferences, allergies, and budget from a database.

[0436] The server generates an optimal meal plan for the user based on the acquired market information and user information. Data analysis algorithms and machine learning models are used to generate the meal plan. Specifically, specific ingredients are selected from market information, and appropriate recipes are extracted based on the user's preferences and allergy information. Through this series of processes, a specific recipe, such as "chicken steamed with cabbage," is suggested.

[0437] The server then creates a shopping list based on the generated recipes and ingredients list, which is then compared with the existing household inventory to eliminate unnecessary items, enabling efficient shopping.

[0438] The user terminal displays the meal plan, cooking instructions, and shopping list received from the server on a user interface. The user can review the information and make corrections as needed. For example, if there is already cabbage at home, the user can remove it from the shopping list.

[0439] A concrete example of this system implementation is when a user wants to make a simple and nutritious meal for a weekday dinner. The user retrieves flyer data for a specific market from their device, checks discounted products, and enters them into the system. The server takes into account the user's preferences and allergy information and suggests, for example, "chicken steamed with cabbage." A detailed recipe and shopping list are generated and displayed on the user interface.

[0440] As an example of a prompt sentence, the following sentence could be fed into a generative AI model:

[0441] "Please explain the system in which a user retrieves flyer data for a specific market from a device, and a server generates and proposes a meal plan to the user based on the user's preferences and allergy information. Specifically, please describe in detail each step from retrieving flyer data to proposing recipes and creating a shopping list."

[0442] In this way, by utilizing generative AI models, it is possible to provide users with efficient and nutritionally balanced meal plans.

[0443] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0444] Step 1:

[0445] The user uses a device to obtain market information. Specifically, the user selects a market, and the device obtains the latest market data through the market information acquisition API. The input here is the market information selected by the user, and the output is market data such as product name, price, discount information, and images. The device temporarily stores this data and sends it to the server.

[0446] Step 2:

[0447] The server retrieves user information. Based on the user ID entered when the user logged in to the application, the server uses an SQL query to retrieve the user's personal profile information from the database. The input is the user ID, and the output is the user's personal profile information, such as preferences, allergies, and budget. This information is stored in the user object within the program.

[0448] Step 3:

[0449] The server generates a meal plan based on market information and user information. The server uses data analysis algorithms and machine learning models to analyze the input market data and user information. The input is market data and user information, and the output is a meal plan suitable for the user (e.g., "chicken steamed with cabbage"). The server extracts related recipes from a home cooking database to suggest more detailed recipes.

[0450] Step 4:

[0451] The server generates detailed cooking instructions based on the meal plan. Specifically, it writes out the ingredient list and cooking instructions from the database and stores the data in a packet in JSON format. The input is the meal plan content, and the output is the ingredient list and detailed cooking instructions (e.g., "200g chicken thighs, 1 / 2 head of cabbage, 1 tomato, salt, pepper" and specific cooking instructions such as "Roughly chop the cabbage. Cut the chicken thighs into bite-sized pieces. Steam the cabbage and chicken thighs.").

[0452] Step 5:

[0453] The server creates a shopping list based on the cooking instructions. The input is the cooking instructions and the household inventory list, and the output is a shopping list with unnecessary items removed. The server compares the existing household inventory list with the newly created ingredient list and removes unnecessary items (e.g., if cabbage is already in the household, remove it from the shopping list).

[0454] Step 6:

[0455] The device displays the created meal plan, cooking instructions, and shopping list to the user. The input is the data received from the server, and the output is the information displayed on the user interface. The device application retrieves this latest data and acts as an interface for the user to review and modify. The user can then review the displayed information, modify it as necessary, and go shopping.

[0456] (Application example 1)

[0457] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0458] Conventional shopping planning support systems have the problem that it takes a lot of time and effort for users to efficiently shop and plan and implement appropriate menus and recipes in their busy daily lives. In particular, the complicated process of generating menus based on the latest flyer information and then ordering the ingredients through a delivery service is a challenge. In addition, the system's inability to provide optimal menu suggestions that meet the user's specific needs can lead to low user satisfaction.

[0459] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0460] In this invention, the server includes means for generating a menu based on flyer data and user information, means for generating recipes based on the generated menu, means for creating a shopping list based on the generated recipe, and means for the terminal to generate prompt sentences using a generative AI model to suggest an optimal menu in response to user requests, and means for providing a delivery service based on the generated shopping list. This enables users to efficiently plan their shopping, smoothly obtain the necessary ingredients through a delivery service, and implement an optimal menu.

[0461] A "user terminal" is a communication device operated by a user, and includes a smartphone, a tablet terminal, a personal computer, and the like.

[0462] "Flyer data" is product information provided by a sales store, and includes data such as product name, price, discount information, and images.

[0463] "User information" is information that indicates the individual characteristics and preferences of a user, including food preferences, allergy information, budget, and the like.

[0464] A "server" is a computer system that processes large amounts of data, and receives and processes data from user terminals.

[0465] The "menu creation means" has the function of creating a specific meal plan based on the acquired flyer data and user information.

[0466] The "recipe generation means" has a function of creating specific cooking procedures and a list of ingredients based on the generated menu.

[0467] The "shopping list creation means" has a function of creating a list of ingredients needed based on the generated recipe.

[0468] The "delivery service providing means" realizes the function of ordering and delivering the necessary ingredients based on the generated shopping list.

[0469] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to suggest optimal menus based on user requests.

[0470] A "prompt sentence" is a sentence input into a generative AI model, and is text that describes the user's requests and conditions.

[0471] The present invention provides a system that combines a user terminal, a server, and a delivery service to efficiently plan shopping and propose menus. Specific embodiments of the system will be described in detail below.

[0472] 1. Hardware Configuration

[0473] In this embodiment, the following hardware is used.

[0474] User devices: smartphones, tablets, personal computers, etc.

[0475] Server: A computer system that processes large amounts of data

[0476] Delivery service: online ordering and delivery system

[0477] 2. Software Configuration

[0478] The system uses the following software:

[0479] Python: Used to develop programs

[0480] Flask: a web framework

[0481] The Requests library: Executing web requests

[0482] SQLite or Firebase: a database for storing user information

[0483] Generative AI model: An artificial intelligence model for proposing optimal menus based on user requests

[0484] 3. Data processing and calculations

[0485] The main data processing and calculation flow performed by the system is shown below.

[0486] (1) Obtaining flyer data

[0487] Users can access the latest flyer data for a designated store from their smartphone or other device. This is done by accessing the store's online service and downloading data such as product name, price, discount information, and images. This information is then sent to a server via the user's device.

[0488] (2) Acquisition of user information

[0489] The server retrieves the user's profile information from a database, including the user's food preferences, allergy information, budget, etc.

[0490] (3) Menu generation

[0491] The server generates the optimal menu based on the acquired flyer data and user information. For example, it selects special offers on chicken thighs, cabbage, tomatoes, etc. from the flyer data and suggests "steamed chicken with cabbage" taking into account the user's preferences and allergies.

[0492] (4) Recipe suggestions

[0493] Once the menu is decided, the server generates a detailed recipe, including a list of specific ingredients (e.g., 200g chicken thighs, 1 / 2 head of cabbage, 1 tomato, salt, and pepper) and cooking instructions (e.g., 1. Roughly shred the cabbage. 2. Cut the chicken thighs into bite-sized pieces. 3. Steam the cabbage and chicken thighs).

[0494] (5) Creating a shopping list

[0495] The server creates a shopping list based on the generated recipe. For example, the list might include "400g of chicken thighs, 1 head of cabbage, and 2 tomatoes." It then compares the list with ingredients already in the home and eliminates any duplicates.

[0496] (6) Providing delivery services

[0497] Implement a means to order the necessary ingredients from a delivery service based on the generated shopping list. This includes the process of ordering ingredients from a user terminal through a delivery service and having them delivered to a specified location.

[0498] 4. Use of generative AI models

[0499] The server uses a generative AI model to propose the optimal menu based on the user's request. Specifically, the AI ​​generates an appropriate menu based on the prompt text entered by the user.

[0500] Prompt Sentence Examples

[0501] "Please suggest a balanced dinner menu based on the user's preferences and the latest flyer information. It should be Japanese, dairy-free, and within a budget of 5,000 yen."

[0502] In this way, the present invention enables users to efficiently plan their shopping, smoothly obtain the necessary ingredients through a delivery service, and implement an optimal menu.

[0503] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0504] Step 1:

[0505] The user terminal acquires the latest flyer data of the designated store.

[0506] Input: User-specified store identification information.

[0507] Operation: The user's device accesses the online service and downloads flyer data such as product names, prices, discount information, and images.

[0508] Output: The retrieved flyer data.

[0509] Step 2:

[0510] The server retrieves the user information from the database.

[0511] Input: User ID.

[0512] How it works: The server accesses a local or cloud database (e.g., SQLite, Firebase) to retrieve information about the user, such as their food preferences, allergies, and budget.

[0513] Output: User information.

[0514] Step 3:

[0515] The server generates the optimal menu based on flyer data and user information.

[0516] Input: Flyer data, user information.

[0517] Operation: The server selects special sale information from flyer data and creates a menu taking into account the user's preferences and allergy information.

[0518] Output: The generated menu.

[0519] Step 4:

[0520] The server generates detailed recipes based on the generated menu.

[0521] Input: The generated menu.

[0522] How it works: The server extracts suitable recipes from a home cooking database and generates specific ingredient lists and cooking instructions.

[0523] Output: A detailed recipe.

[0524] Step 5:

[0525] The server creates a shopping list based on the generated recipe.

[0526] Input: A detailed recipe.

[0527] How it works: The server creates a list of ingredients needed based on the recipe, compares it with the list of ingredients already in the house, and weeds out any duplicates.

[0528] Output: Shopping list.

[0529] Step 6:

[0530] The server provides delivery services based on the shopping list.

[0531] Enter: shopping list.

[0532] How it works: The server uses the delivery service's API to order ingredients and arrange for delivery to the specified location.

[0533] Output: Delivery order confirmation information.

[0534] Step 7:

[0535] The device uses a generative AI model to generate prompts that suggest the optimal menu based on the user's requests.

[0536] Input: User request (e.g., "Please suggest a balanced dinner menu based on the user's preferences and the latest flyer information. Japanese food, dairy-free, within a budget of 5,000 yen.").

[0537] How it works: Enter a prompt into the generative AI model, and the AI ​​will generate an appropriate menu.

[0538] Output: The optimal menu suggested to the user.

[0539] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0540] This invention is a menu suggestion and recipe provision system that combines an emotion engine that recognizes the user's emotions. This system makes the user's eating habits more efficient by having the server generate optimal menus, recipes, and shopping lists based on flyer data, user information, and emotion information acquired by the user's terminal.

[0541] Program processing

[0542] This system is divided into the following main processing steps: Specific operations at each step are as follows:

[0543] 1. Obtain flyer data

[0544] The user requests the download of the latest flyer data for the specified supermarket on their device. The device accesses the supermarket's online service and obtains data such as product names, prices, discount information, and images. The obtained flyer data is then sent to the server.

[0545] 2. Obtaining user information

[0546] The server retrieves the user's personal profile information (preferences, allergy information, budget, etc.) from the database. For example, it confirms that User A's preference is Japanese food, that he has a dairy allergy, and that his weekly food budget is 5,000 yen.

[0547] 3. Acquiring emotional information

[0548] Using an emotion engine built into or connected to the user's device, the system acquires emotional information from the user's facial expressions and tone of voice while they are browsing the flyer. For example, it can recognize emotions such as "tired" or "stressed."

[0549] 4. Menu generation

[0550] The server generates a menu suited to the user based on the acquired flyer data, user information, and emotional information. For example, if the user is tired, it will prioritize simple recipes with short cooking times. It will also select chicken thighs, cabbage, and tomatoes from the flyer data and suggest "chicken steamed with cabbage."

[0551] 5. Recipe suggestions

[0552] The server generates a detailed recipe based on the generated menu, including a list of specific ingredients (e.g., 200g chicken thighs, 1 / 2 head of cabbage, 1 tomato, salt, and pepper) and cooking instructions (e.g., 1. Roughly shred the cabbage. 2. Cut the chicken thighs into bite-sized pieces. 3. Steam the cabbage and chicken thighs).

[0553] 6. Create a shopping list

[0554] The server creates a shopping list based on recipe information. For example, the list might include "400g chicken thighs, 1 head of cabbage, and 2 tomatoes." The server compares the list with the ingredients available at home and removes unnecessary items.

[0555] 7. User Display

[0556] The device displays the meal plan, recipes, and shopping list to the user, who can review and modify them as needed. For example, the device may adjust the display to present particularly simple steps for a tired user.

[0557] Specific examples

[0558] For example, suppose a user finds a sale flyer at a supermarket on a weekday evening and needs to prepare dinner. The user downloads the supermarket flyer data using their device and checks the sale items included in the flyer. The emotion engine recognizes the emotion "tired" from the user's facial expression and tone of voice. The server takes into account the user's preferences and allergy information, as well as the user's emotional fatigue information, and suggests "steamed chicken with cabbage," which is easy to prepare and has excellent nutritional value, as a menu item. A detailed recipe and shopping list are generated and displayed on the user's device. The user can review the list, make any necessary edits, then go shopping at the supermarket and easily prepare the dish at home.

[0559] In this way, the present invention reflects the user's emotional state and provides appropriate menus and recipes, thereby reducing the time and effort required for cooking and easing the burden on the user.

[0560] The processing flow will be explained below.

[0561] Step 1:

[0562] The user makes a request on the terminal to download the latest flyer data of the specified supermarket.

[0563] Step 2:

[0564] The device accesses the online service of the specified supermarket and obtains data such as product name, price, discount information, images, etc. The obtained flyer data is then sent to the server.

[0565] Step 3:

[0566] The server retrieves the user's personal profile information (preferences, allergy information, budget, etc.) from a database.

[0567] Step 4:

[0568] Using an emotion engine built into or connected to the user's device, the system acquires emotional information from the user's facial expressions and tone of voice while they are browsing the flyer. For example, it can recognize emotions such as "tired" or "stressed."

[0569] Step 5:

[0570] The emotion information acquired by the user terminal is transmitted to the server.

[0571] Step 6:

[0572] The server generates a menu suited to the user based on the acquired flyer data, user information, and emotional information. For example, if the user is tired, it will prioritize simple recipes with short cooking times.

[0573] Step 7:

[0574] The server retrieves recipes using the selected ingredients from a home cooking database and generates a detailed recipe, including a specific ingredient list and cooking instructions.

[0575] Step 8:

[0576] The server creates a shopping list based on the generated recipe. For example, the list might include "400g of chicken thighs, 1 head of cabbage, and 2 tomatoes." The server compares the list with the ingredients available at home and removes unnecessary items.

[0577] Step 9:

[0578] The server sends the shopping list to the terminal.

[0579] Step 10:

[0580] The device displays the created menu, recipes, and shopping list to the user, who can review them and make any necessary corrections.

[0581] Step 11:

[0582] The user goes shopping at the supermarket based on the shopping list and cooks food according to the recipe using the purchased ingredients.

[0583] Example 2

[0584] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0585] In recent years, for many users who lead busy lives, preparing meals every day requires time and effort. In particular, it is difficult to propose menus that take into account the user's emotional state and preferences. Therefore, there is a need for menu suggestions that are appropriate for users who are tired or have specific preferences or allergies. There is also a need for support in creating efficient shopping lists and cooking based on those lists. There is a need for systems and methods that can solve these problems.

[0586] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for generating a menu based on flyer data, user information, and emotion information, a means for generating recipes based on the generated menu, and a means for creating a shopping list based on the generated recipe. This makes it possible to provide an optimal menu according to the user's emotions and individual needs, and to save the effort of creating a shopping list and cooking.

[0587] "User terminal" refers to an electronic device such as a computer device or smartphone that is directly operated by a user.

[0588] "Flyer data" is data containing information on the latest products offered by retailers such as supermarkets, and includes product names, prices, discount information, images, and the like.

[0589] "User information" refers to information about a user's personal profile, including preferences, allergy information, budget, and the like.

[0590] "Emotional information" refers to data that indicates the emotional state of the user, such as facial expressions and tone of voice.

[0591] "Menu" refers to the combination of dishes and menus suggested to optimize the user's diet.

[0592] "Recipe" refers to a document or information that includes a specific list of ingredients and cooking instructions based on a menu.

[0593] A "shopping list" is a list of ingredients needed to prepare a recipe, and is intended to support efficient shopping.

[0594] A "server" refers to a computer system that processes requests from terminals via a network and sends and receives necessary data.

[0595] This invention is a menu suggestion and recipe provision system that combines an emotion engine that recognizes the user's emotions. Based on flyer data, user information, and emotion information acquired by the user's terminal, the server generates optimal menus, recipes, and shopping lists and displays them to the user, thereby streamlining the user's eating habits.

[0596] Hardware and software used

[0597] User device: An electronic device that can be directly operated by a user, such as a computer or smartphone, to which a camera, microphone, etc. are connected.

[0598] Server: A computer system that processes requests from terminals and sends and receives data over a network.

[0599] Emotion engine: Software that recognizes emotions from a user's facial expressions and tone of voice. For example, it uses machine learning libraries such as TensorFlow.

[0600] Database: A database system, such as MySQL, for storing user information and flyer data.

[0601] Programming languages ​​and libraries: Python, Pandas, HTTP requests, etc.

[0602] Specific operation explanation

[0603] Obtaining flyer data

[0604] When a user downloads flyer information for a particular supermarket on their device, the device accesses the online service using an HTTP request to obtain data such as product names, prices, discount information, images, etc. This data is temporarily stored in local storage and then sent to the server.

[0605] Obtaining user information

[0606] The server queries the database based on the user ID and obtains information such as the user's preferences, allergies, budget, etc. The data is loaded into memory and handed over to the next process.

[0607] Acquiring emotional information

[0608] The user device uses a camera and microphone to collect facial expressions and tone of voice, which are then analyzed by the emotion engine. TensorFlow is used for emotion recognition, and the estimated results are sent to the server.

[0609] Menu and recipe generation

[0610] The server runs an algorithm to generate optimal menus based on collected flyer data, user information, and emotional information. For example, if the server detects that the user is tired, it prioritizes recipes with short cooking times. This is done using Python scripts and the Pandas library. From the generated menu, a detailed recipe is created, including a specific ingredient list and cooking instructions.

[0611] Creating a shopping list

[0612] Based on the generated recipe, the necessary ingredients are compiled into a shopping list, which is then compared with the ingredients in your home and unnecessary items are removed to create the final shopping list.

[0613] What the user sees

[0614] Finally, the generated menu, detailed recipes, and shopping list are displayed on the user's device, where the user can review the information provided and make any necessary corrections.

[0615] Prompt Sentence Examples

[0616] For example, an example of a prompt sentence to be input into a generative AI model is, "When a user finds a sale flyer at the supermarket on a weekday evening and needs to prepare dinner, please explain the process of the system recognizing the user's tiredness from their facial expression and tone of voice, and suggesting a menu that is easy to prepare while taking those emotions into consideration."

[0617] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0618] Step 1: Obtain flyer data

[0619] User: The user operates the device and issues a request to download flyer data for a specific supermarket.

[0620] Input: Supermarket flyer URL

[0621] Terminal: Upon receiving a request from the user, the terminal uses an HTTP request to access the supermarket's online service and retrieve flyer data.

[0622] Data processing: The acquired data includes product names, prices, discount information, images, etc. This data is parsed, organized, and saved.

[0623] Output: Organized flyer data

[0624] Device: The organized flyer data is temporarily stored in local storage, and then sent to the server via an HTTP POST request.

[0625] Step 2: Get user information

[0626] Server: Queries the database based on the user ID and retrieves the user's personal profile information.

[0627] Input: User ID

[0628] Data processing: Information retrieved from the database includes user preferences, allergies, budget, etc. This information is loaded into memory and organized for further processing.

[0629] Output: User's personal profile information

[0630] Server: Records the acquired user information in a log and passes it to the subsequent menu generation process.

[0631] Step 3: Acquiring emotional information

[0632] Device: Uses a camera and microphone to capture your facial expressions and tone of voice.

[0633] Input: User facial and voice data

[0634] Data processing: Using an emotion recognition engine (e.g., TensorFlow), the collected data is analyzed to estimate the user's emotional state, and emotional information such as fatigue and stress levels is obtained.

[0635] Output: User's emotional information

[0636] Device: Sends analyzed emotion information to the server.

[0637] Step 4: Create a menu

[0638] Server: Runs an algorithm to generate a menu suitable for the user based on the acquired flyer data, user information, and emotional information.

[0639] Input: flyer data, user information, emotion information

[0640] Data processing: Using algorithms, recipe information stored in the database is filtered based on evaluation criteria (ease of use, nutritional balance, cooking time). Based on emotional information, for example, recipes with short cooking times are prioritized for tired users.

[0641] Output: Optimal meal plan

[0642] Server: Pass the generated menu plan to the next recipe generation step.

[0643] Step 5: Recipe suggestions

[0644] Server: Runs a script that generates detailed recipes based on optimal menu plans.

[0645] Input: Optimal meal plan

[0646] Data processing: Using the Pandas library, generate a specific list of ingredients (e.g., 200g chicken thighs, 1 / 2 head of cabbage, 1 tomato, salt, and pepper) and cooking instructions (e.g., roughly chop the cabbage. Cut the chicken thighs into bite-sized pieces. Steam the cabbage and chicken thighs).

[0647] Output: Detailed recipe

[0648] Server: Pass the generated recipe to the shopping list generation step.

[0649] Step 6: Create a shopping list

[0650] Server: Based on the recipe information, it creates a list of the necessary ingredients and creates a shopping list.

[0651] Input: Detailed recipe

[0652] Data processing: Match the data with the grocery list at home, filter out unnecessary items, and generate the final shopping list.

[0653] Output: Final shopping list

[0654] Server: Sends the generated shopping list to the device.

[0655] Step 7: Display to the user

[0656] Terminal: Displays the menu, recipes, and shopping list received from the server on the user interface.

[0657] Input: Menus, recipes, shopping lists

[0658] Data processing: Organize and display information in a format that is easy for users to view. If the user makes any corrections, the data is sent back to the server.

[0659] Output: Menu, recipes, and shopping list displayed to the user

[0660] User: Review the information provided and make corrections as needed.

[0661] (Application example 2)

[0662] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0663] In modern society, as user lifestyles become more diverse, systems that suggest optimal meals based on an individual's emotional and health states are becoming increasingly important. However, conventional systems often suggest menus based solely on the user's preferences and budget, without taking the user's emotional state into consideration, resulting in low user satisfaction and low utilization rates. Furthermore, for busy modern people, finding the optimal meal for their emotional and physical states can be a significant burden. Therefore, there is a need for a system that utilizes the user's emotional information to suggest optimal menus.

[0664] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for acquiring flyer data, a means for acquiring emotion information, a means for generating a menu based on user information, a means for generating recipes based on the generated menu, and a means for creating a shopping list based on the generated recipe. This makes it possible to propose an optimal menu that reflects the user's emotional state.

[0665] A "user terminal" is an electronic device used by a user, and includes smartphones, tablets, PCs, etc.

[0666] "Flyer data" is advertising material issued by supermarkets and stores, and includes product names, prices, discount information, images, and the like.

[0667] "Emotional information" is data on the user's emotional state based on facial expressions, tone of voice, and other biological information.

[0668] "User information" refers to personal profile information such as the user's preferences, allergy information, budget, etc.

[0669] A "server" is a central system that processes and stores information on a network and communicates with clients (user terminals).

[0670] A "menu" is a meal plan made up of a combination of multiple dishes and ingredients.

[0671] A "recipe" is a list of ingredients and specific steps for making a dish.

[0672] A "shopping list" is a list of ingredients and products needed based on a recipe.

[0673] A "system" is a collection of devices and programs in which multiple functional components work in conjunction with one another.

[0674] This invention is a system that utilizes user emotion information to propose optimal menus for food delivery. This system includes a user terminal, an emotion engine, a server, and a network for communication among these.

[0675] System configuration

[0676] The user terminal is an electronic device such as a smartphone, tablet, or PC through which the user accesses the system. The user terminal has an internet connection function and allows input of the emotion engine and user information.

[0677] The emotion engine is software or hardware that analyzes a user's facial expressions, tone of voice, and other biometric information to generate emotional information about the user. The emotion engine is either built into the user terminal or operates as an external device connected to the user terminal.

[0678] The server is a central system that processes and stores emotion information, flyer data, and user information over the network. The server manages data such as user profiles, flyer databases, and recipe databases, and generates optimal menus and recipes for users.

[0679] System Operation

[0680] The operation of the system is as follows.

[0681] 1. Obtain flyer data:

[0682] The user's device downloads the latest menu information for the specified food delivery service and transmits it to the server, including product names, prices, images, etc.

[0683] 2. Acquiring emotional information:

[0684] The user device uses an emotion engine to obtain emotional information (e.g., "I'm tired" or "I'm stressed") from the user's facial expressions and tone of voice, and sends it to the server.

[0685] 3. Obtaining user information:

[0686] The server retrieves personal profile information such as the user's preferences, allergy information, budget, etc. from the database. For example, it confirms that the user's preference is Japanese food, that they have a dairy allergy, and that their weekly food budget is 5,000 yen.

[0687] 4. Menu generation:

[0688] The server generates a menu tailored to the user based on the flyer data, emotional information, and user information it has acquired. For example, if the user is tired, it will prioritize and suggest simple menus that require less time to cook.

[0689] 5. Recipe generation and shopping list creation:

[0690] Based on the generated menu, the server generates a detailed recipe and a shopping list for its execution, including a specific list of ingredients and cooking instructions.

[0691] 6. User Visibility:

[0692] The user terminal displays the menu, recipes, and shopping list sent from the server, and the user can check them and place orders as necessary.

[0693] Specific examples

[0694] Suppose a user is browsing a food delivery menu on a weekday evening. The user's device downloads the latest menu information (product name, price, and image), and the emotion engine recognizes the user's emotion, "tired," from their facial expression and tone of voice. The server takes this emotional information, along with the user's preferences and allergies, into consideration to suggest a menu that is easy to prepare and nutritious. For example, it suggests a dish called "chicken steamed with cabbage," and a detailed recipe and list of ingredients are displayed on the user's device.

[0695] Prompt Sentence Examples

[0696] "The user is extremely tired. Please suggest a menu that you would recommend at this time."

[0697] "Emotional information: I'm tired. Please suggest a menu that requires little cooking time and is within a budget of 5,000 yen."

[0698] In this way, it becomes possible to propose an optimal menu that reflects the user's emotional state.

[0699] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0700] Step 1:

[0701] The user device obtains the latest menu information from a specified food delivery service. Specifically, the user device accesses the food delivery service's API via the Internet and downloads menu information (product name, price, image, etc.). This menu information is sent from the user device to a server for use in subsequent processing steps. The input is the menu information from the food delivery service API, and the output is the menu information sent to the server.

[0702] Step 2:

[0703] The user device uses an emotion engine to obtain the user's emotional information. Specifically, the emotion engine analyzes the user's facial expression and tone of voice in real time and generates emotional information (for example, "tired" or "stressed"). This emotional information is sent from the user device to the server. The input is the user's facial expression and tone of voice, and the output is the emotional information sent to the server.

[0704] Step 3:

[0705] The server retrieves the user's personal profile information from a database, including preferences, allergies, budget, etc. This information is then combined with other data to generate an optimal menu. The input is a query to the server's database, and the output is the retrieved user information.

[0706] Step 4:

[0707] The server generates a menu suitable for the user based on the acquired flyer data, emotional information, and user information. Specifically, it prioritizes menus with short cooking times and menus with a relaxing effect based on emotional information (e.g., "tired"). The inputs are flyer data, emotional information, and user information, and the output is the generated menu.

[0708] Step 5:

[0709] The server generates a detailed recipe and a shopping list based on the generated menu. Specifically, the recipe includes a list of specific ingredients (e.g., 200g chicken thighs, 1 / 2 head of cabbage, 1 tomato) and cooking instructions (e.g., 1. Roughly chop the cabbage. 2. Cut the chicken thighs into bite-sized pieces), and the shopping list lists the ingredients needed based on the recipe. The input is the generated menu, and the output is a detailed recipe and a shopping list.

[0710] Step 6:

[0711] The user terminal displays the menu, recipes, and shopping list sent from the server. Specifically, the menu, recipes, and shopping list are displayed on the screen of the user terminal, and the user checks them and places an order as necessary. The system also encourages the user to select a menu based on their emotions when ordering. The input is the data sent from the server, and the output is the menu, recipes, and shopping list displayed on the user terminal.

[0712] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0713] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0714] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0715] [Third embodiment]

[0716] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0717] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0718] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0719] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0720] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0721] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0722] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0723] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0724] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0725] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0726] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0727] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0728] This invention is a system in which a server generates and proposes menus to users based on flyer data and user information acquired by the user terminal. This system helps users to shop efficiently and plan and implement appropriate menus and recipes even in their busy daily lives.

[0729] Program processing

[0730] This system is divided into the following main processing steps: Specific operations at each step are as follows:

[0731] 1. Obtain flyer data

[0732] A user uses a device to download the latest flyer data for a specified supermarket. For example, the user selects a flyer for Supermarket X. The device accesses Supermarket X's online service, obtains data such as product name, price, discount information, and images, and sends this data to the server.

[0733] 2. Obtaining user information

[0734] The server retrieves the user's personal profile information from the database. For example, it confirms that User A's preference is Japanese food, that he / she has a dairy allergy, and that his / her weekly food budget is 5,000 yen.

[0735] 3. Menu generation

[0736] The server generates a menu tailored to the user based on the flyer data and user information it has acquired. For example, it selects chicken thighs, cabbage, and tomatoes from the flyer data, extracts the user's favorite Japanese recipe from a home cooking database, and suggests "chicken steamed with cabbage."

[0737] 4. Recipe suggestions

[0738] The server generates a detailed recipe based on the menu, including a list of specific ingredients (e.g., 200g chicken thighs, 1 / 2 head of cabbage, 1 tomato, salt, and pepper) and cooking instructions (e.g., 1. Roughly shred the cabbage. 2. Cut the chicken thighs into bite-sized pieces. 3. Steam the cabbage and chicken thighs).

[0739] 5. Create a shopping list

[0740] The server creates a shopping list based on recipe information. For example, the list might include "400g chicken thighs, 1 head of cabbage, and 2 tomatoes." The server compares the list with the ingredients available at home and removes unnecessary items.

[0741] 6. User Display

[0742] The device displays the created menu, recipes, and shopping list to the user. The user can review this information and make corrections as necessary. For example, the user may decide that they do not need to buy cabbage because they already have it at home and correct the list accordingly.

[0743] Specific examples

[0744] For example, suppose a user wants to make a simple, nutritious meal for a weekday dinner. The user downloads a flyer for Supermarket X from their device and checks the specials it contains. They learn that chicken thighs, cabbage, and tomatoes are on sale, and use the system to suggest a meal. The server considers the user's preferences and allergy information and suggests "steamed chicken with cabbage." A detailed recipe is generated, and a shopping list is also created. The user can review this information on their device, make any necessary corrections, and then go shopping. In this way, users can efficiently plan their meals and ensure the shopping and cooking processes proceed smoothly.

[0745] The processing flow will be explained below.

[0746] Step 1:

[0747] The user makes a request on the terminal to download the latest flyer data of the specified supermarket.

[0748] Step 2:

[0749] The device accesses the supermarket's online service and retrieves the latest flyer data (product name, price, discount information, images, etc.).

[0750] Step 3:

[0751] The terminal transmits the acquired flyer data to the server.

[0752] Step 4:

[0753] The server retrieves the user's personal profile information (preferences, allergy information, budget, etc.) from a database.

[0754] Step 5:

[0755] The server generates a menu tailored to the user based on the acquired flyer data and user information.

[0756] Step 6:

[0757] The server retrieves recipes using the selected ingredients from a home cooking database and generates a detailed recipe, including a specific ingredient list and cooking instructions.

[0758] Step 7:

[0759] The server creates a shopping list of the necessary ingredients based on the generated recipe.

[0760] Step 8:

[0761] The server sends the shopping list to the terminal.

[0762] Step 9:

[0763] The device displays the created menu, recipes, and shopping list to the user, who can review and modify them as necessary.

[0764] Step 10:

[0765] The user goes shopping at the supermarket based on the shopping list and cooks using the purchased ingredients according to the recipe.

[0766] Example 1

[0767] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0768] Modern users face the challenge of efficiently shopping and planning and implementing appropriate menus and recipes in their busy daily lives. Conventional methods require users to collect market information and manually plan menus tailored to their individual needs, which takes a lot of time and effort. For this reason, there is a demand for a system that allows users to easily plan and implement nutritionally balanced meals while efficiently purchasing ingredients.

[0769] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0770] In this invention, the server includes means for generating a meal plan based on market information and user information, means for generating cooking instructions based on the generated meal plan, and means for creating a shopping list based on the generated cooking instructions. This allows users to automatically receive market information and meal plans and recipes tailored to their individual needs, allowing them to shop efficiently and easily plan and carry out appropriate meals.

[0771] A "user terminal" is an electronic device that allows a user to input and receive information.

[0772] "Market information" is data on products sold in specific markets or stores, and includes product names, prices, discount information, images, and the like.

[0773] "User information" is data about an individual user, including preferences, allergy information, budget, and the like.

[0774] A "meal plan" is a menu created based on the user's market information and user information.

[0775] "Cooking Instructions" are detailed instructions and ingredient lists for dishes based on a meal plan.

[0776] A "shopping list" is a list of ingredients and items needed based on the cooking procedure.

[0777] A "system" is a collection of hardware and software, including user terminals and servers, for realizing specific functions.

[0778] A "server" is a centralized computer system that processes and stores data.

[0779] This invention is a system that generates meal plans based on market information and user information and proposes them to users. This system helps users shop efficiently and plan and implement appropriate menus and recipes.

[0780] The system consists of user devices, servers, and supporting hardware and software. Specifically, it includes user devices such as smartphones and tablets, a server with a database, and an internet connection. The software supporting this hardware includes a market information acquisition API, a user profile management system, data analysis algorithms, machine learning models, and a user interface application.

[0781] Users can operate the application using a smartphone or tablet to obtain the latest information on a specific market. Market information includes product names, prices, discount information, images, etc. This information is sent to the server through a market information acquisition API. The server then retrieves personal profile information such as the user's preferences, allergies, and budget from a database.

[0782] The server generates an optimal meal plan for the user based on the acquired market information and user information. Data analysis algorithms and machine learning models are used to generate the meal plan. Specifically, specific ingredients are selected from market information, and appropriate recipes are extracted based on the user's preferences and allergy information. Through this series of processes, a specific recipe, such as "chicken steamed with cabbage," is suggested.

[0783] The server then creates a shopping list based on the generated recipes and ingredients list, which is then compared with the existing household inventory to eliminate unnecessary items, enabling efficient shopping.

[0784] The user terminal displays the meal plan, cooking instructions, and shopping list received from the server on a user interface. The user can review the information and make corrections as needed. For example, if there is already cabbage at home, the user can remove it from the shopping list.

[0785] A concrete example of this system implementation is when a user wants to make a simple and nutritious meal for a weekday dinner. The user retrieves flyer data for a specific market from their device, checks discounted products, and enters them into the system. The server takes into account the user's preferences and allergy information and suggests, for example, "chicken steamed with cabbage." A detailed recipe and shopping list are generated and displayed on the user interface.

[0786] As an example of a prompt sentence, the following sentence could be fed into a generative AI model:

[0787] "Please explain the system in which a user retrieves flyer data for a specific market from a device, and a server generates and proposes a meal plan to the user based on the user's preferences and allergy information. Specifically, please describe in detail each step from retrieving flyer data to proposing recipes and creating a shopping list."

[0788] In this way, by utilizing generative AI models, it is possible to provide users with efficient and nutritionally balanced meal plans.

[0789] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0790] Step 1:

[0791] The user uses a device to obtain market information. Specifically, the user selects a market, and the device obtains the latest market data through the market information acquisition API. The input here is the market information selected by the user, and the output is market data such as product name, price, discount information, and images. The device temporarily stores this data and sends it to the server.

[0792] Step 2:

[0793] The server retrieves user information. Based on the user ID entered when the user logged in to the application, the server uses an SQL query to retrieve the user's personal profile information from the database. The input is the user ID, and the output is the user's personal profile information, such as preferences, allergies, and budget. This information is stored in the user object within the program.

[0794] Step 3:

[0795] The server generates a meal plan based on market information and user information. The server uses data analysis algorithms and machine learning models to analyze the input market data and user information. The input is market data and user information, and the output is a meal plan suitable for the user (e.g., "chicken steamed with cabbage"). The server extracts related recipes from a home cooking database to suggest more detailed recipes.

[0796] Step 4:

[0797] The server generates detailed cooking instructions based on the meal plan. Specifically, it writes out the ingredient list and cooking instructions from the database and stores the data in a packet in JSON format. The input is the meal plan content, and the output is the ingredient list and detailed cooking instructions (e.g., "200g chicken thighs, 1 / 2 head of cabbage, 1 tomato, salt, pepper" and specific cooking instructions such as "Roughly chop the cabbage. Cut the chicken thighs into bite-sized pieces. Steam the cabbage and chicken thighs.").

[0798] Step 5:

[0799] The server creates a shopping list based on the cooking instructions. The input is the cooking instructions and the household inventory list, and the output is a shopping list with unnecessary items removed. The server compares the existing household inventory list with the newly created ingredient list and removes unnecessary items (e.g., if cabbage is already in the household, remove it from the shopping list).

[0800] Step 6:

[0801] The device displays the created meal plan, cooking instructions, and shopping list to the user. The input is the data received from the server, and the output is the information displayed on the user interface. The device application retrieves this latest data and acts as an interface for the user to review and modify. The user can then review the displayed information, modify it as necessary, and go shopping.

[0802] (Application example 1)

[0803] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0804] Conventional shopping planning support systems have the problem that it takes a lot of time and effort for users to efficiently shop and plan and implement appropriate menus and recipes in their busy daily lives. In particular, the complicated process of generating menus based on the latest flyer information and then ordering the ingredients through a delivery service is a challenge. In addition, the system's inability to provide optimal menu suggestions that meet the user's specific needs can lead to low user satisfaction.

[0805] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0806] In this invention, the server includes means for generating a menu based on flyer data and user information, means for generating recipes based on the generated menu, means for creating a shopping list based on the generated recipe, and means for the terminal to generate prompt sentences using a generative AI model to suggest an optimal menu in response to user requests, and means for providing a delivery service based on the generated shopping list. This enables users to efficiently plan their shopping, smoothly obtain the necessary ingredients through a delivery service, and implement an optimal menu.

[0807] A "user terminal" is a communication device operated by a user, and includes a smartphone, a tablet terminal, a personal computer, and the like.

[0808] "Flyer data" is product information provided by a sales store, and includes data such as product name, price, discount information, and images.

[0809] "User information" is information that indicates the individual characteristics and preferences of a user, including food preferences, allergy information, budget, and the like.

[0810] A "server" is a computer system that processes large amounts of data, and receives and processes data from user terminals.

[0811] The "menu creation means" has the function of creating a specific meal plan based on the acquired flyer data and user information.

[0812] The "recipe generation means" has a function of creating specific cooking procedures and a list of ingredients based on the generated menu.

[0813] The "shopping list creation means" has a function of creating a list of ingredients needed based on the generated recipe.

[0814] The "delivery service providing means" realizes the function of ordering and delivering the necessary ingredients based on the generated shopping list.

[0815] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to suggest optimal menus based on user requests.

[0816] A "prompt sentence" is a sentence input into a generative AI model, and is text that describes the user's requests and conditions.

[0817] The present invention provides a system that combines a user terminal, a server, and a delivery service to efficiently plan shopping and propose menus. Specific embodiments of the system will be described in detail below.

[0818] 1. Hardware Configuration

[0819] In this embodiment, the following hardware is used.

[0820] User devices: smartphones, tablets, personal computers, etc.

[0821] Server: A computer system that processes large amounts of data

[0822] Delivery service: online ordering and delivery system

[0823] 2. Software Configuration

[0824] The system uses the following software:

[0825] Python: Used to develop programs

[0826] Flask: a web framework

[0827] The Requests library: Executing web requests

[0828] SQLite or Firebase: a database for storing user information

[0829] Generative AI model: An artificial intelligence model for proposing optimal menus based on user requests

[0830] 3. Data processing and calculations

[0831] The main data processing and calculation flow performed by the system is shown below.

[0832] (1) Obtaining flyer data

[0833] Users can access the latest flyer data for a designated store from their smartphone or other device. This is done by accessing the store's online service and downloading data such as product name, price, discount information, and images. This information is then sent to a server via the user's device.

[0834] (2) Acquisition of user information

[0835] The server retrieves the user's profile information from a database, including the user's food preferences, allergy information, budget, etc.

[0836] (3) Menu generation

[0837] The server generates the optimal menu based on the acquired flyer data and user information. For example, it selects special offers on chicken thighs, cabbage, tomatoes, etc. from the flyer data and suggests "steamed chicken with cabbage" taking into account the user's preferences and allergies.

[0838] (4) Recipe suggestions

[0839] Once the menu is decided, the server generates a detailed recipe, including a list of specific ingredients (e.g., 200g chicken thighs, 1 / 2 head of cabbage, 1 tomato, salt, and pepper) and cooking instructions (e.g., 1. Roughly shred the cabbage. 2. Cut the chicken thighs into bite-sized pieces. 3. Steam the cabbage and chicken thighs).

[0840] (5) Creating a shopping list

[0841] The server creates a shopping list based on the generated recipe. For example, the list might include "400g of chicken thighs, 1 head of cabbage, and 2 tomatoes." It then compares the list with ingredients already in the home and eliminates any duplicates.

[0842] (6) Providing delivery services

[0843] Implement a means to order the necessary ingredients from a delivery service based on the generated shopping list. This includes the process of ordering ingredients from a user terminal through a delivery service and having them delivered to a specified location.

[0844] 4. Use of generative AI models

[0845] The server uses a generative AI model to propose the optimal menu based on the user's request. Specifically, the AI ​​generates an appropriate menu based on the prompt text entered by the user.

[0846] Prompt Sentence Examples

[0847] "Please suggest a balanced dinner menu based on the user's preferences and the latest flyer information. It should be Japanese, dairy-free, and within a budget of 5,000 yen."

[0848] In this way, the present invention enables users to efficiently plan their shopping, smoothly obtain the necessary ingredients through a delivery service, and implement an optimal menu.

[0849] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0850] Step 1:

[0851] The user terminal acquires the latest flyer data of the designated store.

[0852] Input: User-specified store identification information.

[0853] Operation: The user's device accesses the online service and downloads flyer data such as product names, prices, discount information, and images.

[0854] Output: The retrieved flyer data.

[0855] Step 2:

[0856] The server retrieves the user information from the database.

[0857] Input: User ID.

[0858] How it works: The server accesses a local or cloud database (e.g., SQLite, Firebase) to retrieve information about the user, such as their food preferences, allergies, and budget.

[0859] Output: User information.

[0860] Step 3:

[0861] The server generates the optimal menu based on flyer data and user information.

[0862] Input: Flyer data, user information.

[0863] Operation: The server selects special sale information from flyer data and creates a menu taking into account the user's preferences and allergy information.

[0864] Output: The generated menu.

[0865] Step 4:

[0866] The server generates detailed recipes based on the generated menu.

[0867] Input: The generated menu.

[0868] How it works: The server extracts suitable recipes from a home cooking database and generates specific ingredient lists and cooking instructions.

[0869] Output: A detailed recipe.

[0870] Step 5:

[0871] The server creates a shopping list based on the generated recipe.

[0872] Input: A detailed recipe.

[0873] How it works: The server creates a list of ingredients needed based on the recipe, compares it with the list of ingredients already in the house, and weeds out any duplicates.

[0874] Output: Shopping list.

[0875] Step 6:

[0876] The server provides delivery services based on the shopping list.

[0877] Enter: shopping list.

[0878] How it works: The server uses the delivery service's API to order ingredients and arrange for delivery to the specified location.

[0879] Output: Delivery order confirmation information.

[0880] Step 7:

[0881] The device uses a generative AI model to generate prompts that suggest the optimal menu based on the user's requests.

[0882] Input: User request (e.g., "Please suggest a balanced dinner menu based on the user's preferences and the latest flyer information. Japanese food, dairy-free, within a budget of 5,000 yen.").

[0883] How it works: Enter a prompt into the generative AI model, and the AI ​​will generate an appropriate menu.

[0884] Output: The optimal menu suggested to the user.

[0885] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0886] This invention is a menu suggestion and recipe provision system that combines an emotion engine that recognizes the user's emotions. This system makes the user's eating habits more efficient by having the server generate optimal menus, recipes, and shopping lists based on flyer data, user information, and emotion information acquired by the user's terminal.

[0887] Program processing

[0888] This system is divided into the following main processing steps: Specific operations at each step are as follows:

[0889] 1. Obtain flyer data

[0890] The user requests the download of the latest flyer data for the specified supermarket on their device. The device accesses the supermarket's online service and obtains data such as product names, prices, discount information, and images. The obtained flyer data is then sent to the server.

[0891] 2. Obtaining user information

[0892] The server retrieves the user's personal profile information (preferences, allergy information, budget, etc.) from the database. For example, it confirms that User A's preference is Japanese food, that he has a dairy allergy, and that his weekly food budget is 5,000 yen.

[0893] 3. Acquiring emotional information

[0894] Using an emotion engine built into or connected to the user's device, the system acquires emotional information from the user's facial expressions and tone of voice while they are browsing the flyer. For example, it can recognize emotions such as "tired" or "stressed."

[0895] 4. Menu generation

[0896] The server generates a menu suited to the user based on the acquired flyer data, user information, and emotional information. For example, if the user is tired, it will prioritize simple recipes with short cooking times. It will also select chicken thighs, cabbage, and tomatoes from the flyer data and suggest "chicken steamed with cabbage."

[0897] 5. Recipe suggestions

[0898] The server generates a detailed recipe based on the generated menu, including a list of specific ingredients (e.g., 200g chicken thighs, 1 / 2 head of cabbage, 1 tomato, salt, and pepper) and cooking instructions (e.g., 1. Roughly shred the cabbage. 2. Cut the chicken thighs into bite-sized pieces. 3. Steam the cabbage and chicken thighs).

[0899] 6. Create a shopping list

[0900] The server creates a shopping list based on recipe information. For example, the list might include "400g chicken thighs, 1 head of cabbage, and 2 tomatoes." The server compares the list with the ingredients available at home and removes unnecessary items.

[0901] 7. User Display

[0902] The device displays the meal plan, recipes, and shopping list to the user, who can review and modify them as needed. For example, the device may adjust the display to present particularly simple steps for a tired user.

[0903] Specific examples

[0904] For example, suppose a user finds a sale flyer at a supermarket on a weekday evening and needs to prepare dinner. The user downloads the supermarket flyer data using their device and checks the sale items included in the flyer. The emotion engine recognizes the emotion "tired" from the user's facial expression and tone of voice. The server takes into account the user's preferences and allergy information, as well as the user's emotional fatigue information, and suggests "steamed chicken with cabbage," which is easy to prepare and has excellent nutritional value, as a menu item. A detailed recipe and shopping list are generated and displayed on the user's device. The user can review the list, make any necessary edits, then go shopping at the supermarket and easily prepare the dish at home.

[0905] In this way, the present invention reflects the user's emotional state and provides appropriate menus and recipes, thereby reducing the time and effort required for cooking and easing the burden on the user.

[0906] The processing flow will be explained below.

[0907] Step 1:

[0908] The user makes a request on the terminal to download the latest flyer data of the specified supermarket.

[0909] Step 2:

[0910] The device accesses the online service of the specified supermarket and obtains data such as product name, price, discount information, images, etc. The obtained flyer data is then sent to the server.

[0911] Step 3:

[0912] The server retrieves the user's personal profile information (preferences, allergy information, budget, etc.) from a database.

[0913] Step 4:

[0914] Using an emotion engine built into or connected to the user's device, the system acquires emotional information from the user's facial expressions and tone of voice while they are browsing the flyer. For example, it can recognize emotions such as "tired" or "stressed."

[0915] Step 5:

[0916] The emotion information acquired by the user terminal is transmitted to the server.

[0917] Step 6:

[0918] The server generates a menu suited to the user based on the acquired flyer data, user information, and emotional information. For example, if the user is tired, it will prioritize simple recipes with short cooking times.

[0919] Step 7:

[0920] The server retrieves recipes using the selected ingredients from a home cooking database and generates a detailed recipe, including a specific ingredient list and cooking instructions.

[0921] Step 8:

[0922] The server creates a shopping list based on the generated recipe. For example, the list might include "400g of chicken thighs, 1 head of cabbage, and 2 tomatoes." The server compares the list with the ingredients available at home and removes unnecessary items.

[0923] Step 9:

[0924] The server sends the shopping list to the terminal.

[0925] Step 10:

[0926] The device displays the created menu, recipes, and shopping list to the user, who can review them and make any necessary corrections.

[0927] Step 11:

[0928] The user goes shopping at the supermarket based on the shopping list and cooks food according to the recipe using the purchased ingredients.

[0929] Example 2

[0930] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0931] In recent years, for many users who lead busy lives, preparing meals every day requires time and effort. In particular, it is difficult to propose menus that take into account the user's emotional state and preferences. Therefore, there is a need for menu suggestions that are appropriate for users who are tired or have specific preferences or allergies. There is also a need for support in creating efficient shopping lists and cooking based on those lists. There is a need for systems and methods that can solve these problems.

[0932] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for generating a menu based on flyer data, user information, and emotion information, a means for generating recipes based on the generated menu, and a means for creating a shopping list based on the generated recipe. This makes it possible to provide an optimal menu according to the user's emotions and individual needs, and to save the effort of creating a shopping list and cooking.

[0933] "User terminal" refers to an electronic device such as a computer device or smartphone that is directly operated by a user.

[0934] "Flyer data" is data containing information on the latest products offered by retailers such as supermarkets, and includes product names, prices, discount information, images, and the like.

[0935] "User information" refers to information about a user's personal profile, including preferences, allergy information, budget, and the like.

[0936] "Emotional information" refers to data that indicates the emotional state of the user, such as facial expressions and tone of voice.

[0937] "Menu" refers to the combination of dishes and menus suggested to optimize the user's diet.

[0938] "Recipe" refers to a document or information that includes a specific list of ingredients and cooking instructions based on a menu.

[0939] A "shopping list" is a list of ingredients needed to prepare a recipe, and is intended to support efficient shopping.

[0940] A "server" refers to a computer system that processes requests from terminals via a network and sends and receives necessary data.

[0941] This invention is a menu suggestion and recipe provision system that combines an emotion engine that recognizes the user's emotions. Based on flyer data, user information, and emotion information acquired by the user's terminal, the server generates optimal menus, recipes, and shopping lists and displays them to the user, thereby streamlining the user's eating habits.

[0942] Hardware and software used

[0943] User device: An electronic device that can be directly operated by a user, such as a computer or smartphone, to which a camera, microphone, etc. are connected.

[0944] Server: A computer system that processes requests from terminals and sends and receives data over a network.

[0945] Emotion engine: Software that recognizes emotions from a user's facial expressions and tone of voice. For example, it uses machine learning libraries such as TensorFlow.

[0946] Database: A database system, such as MySQL, for storing user information and flyer data.

[0947] Programming languages ​​and libraries: Python, Pandas, HTTP requests, etc.

[0948] Specific operation explanation

[0949] Obtaining flyer data

[0950] When a user downloads flyer information for a particular supermarket on their device, the device accesses the online service using an HTTP request to obtain data such as product names, prices, discount information, images, etc. This data is temporarily stored in local storage and then sent to the server.

[0951] Obtaining user information

[0952] The server queries the database based on the user ID and obtains information such as the user's preferences, allergies, budget, etc. The data is loaded into memory and handed over to the next process.

[0953] Acquiring emotional information

[0954] The user device uses a camera and microphone to collect facial expressions and tone of voice, which are then analyzed by the emotion engine. TensorFlow is used for emotion recognition, and the estimated results are sent to the server.

[0955] Menu and recipe generation

[0956] The server runs an algorithm to generate optimal menus based on collected flyer data, user information, and emotional information. For example, if the server detects that the user is tired, it prioritizes recipes with short cooking times. This is done using Python scripts and the Pandas library. From the generated menu, a detailed recipe is created, including a specific ingredient list and cooking instructions.

[0957] Creating a shopping list

[0958] Based on the generated recipe, the necessary ingredients are compiled into a shopping list, which is then compared with the ingredients in your home and unnecessary items are removed to create the final shopping list.

[0959] What the user sees

[0960] Finally, the generated menu, detailed recipes, and shopping list are displayed on the user's device, where the user can review the information provided and make any necessary corrections.

[0961] Prompt Sentence Examples

[0962] For example, an example of a prompt sentence to be input into a generative AI model is, "When a user finds a sale flyer at the supermarket on a weekday evening and needs to prepare dinner, please explain the process of the system recognizing the user's tiredness from their facial expression and tone of voice, and suggesting a menu that is easy to prepare while taking those emotions into consideration."

[0963] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0964] Step 1: Obtain flyer data

[0965] User: The user operates the device and issues a request to download flyer data for a specific supermarket.

[0966] Input: Supermarket flyer URL

[0967] Terminal: Upon receiving a request from the user, the terminal uses an HTTP request to access the supermarket's online service and retrieve flyer data.

[0968] Data processing: The acquired data includes product names, prices, discount information, images, etc. This data is parsed, organized, and saved.

[0969] Output: Organized flyer data

[0970] Device: The organized flyer data is temporarily stored in local storage, and then sent to the server via an HTTP POST request.

[0971] Step 2: Get user information

[0972] Server: Queries the database based on the user ID and retrieves the user's personal profile information.

[0973] Input: User ID

[0974] Data processing: Information retrieved from the database includes user preferences, allergies, budget, etc. This information is loaded into memory and organized for further processing.

[0975] Output: User's personal profile information

[0976] Server: Records the acquired user information in a log and passes it to the subsequent menu generation process.

[0977] Step 3: Acquiring emotional information

[0978] Device: Uses a camera and microphone to capture your facial expressions and tone of voice.

[0979] Input: User facial and voice data

[0980] Data processing: Using an emotion recognition engine (e.g., TensorFlow), the collected data is analyzed to estimate the user's emotional state, and emotional information such as fatigue and stress levels is obtained.

[0981] Output: User's emotional information

[0982] Device: Sends analyzed emotion information to the server.

[0983] Step 4: Create a menu

[0984] Server: Runs an algorithm to generate a menu suitable for the user based on the acquired flyer data, user information, and emotional information.

[0985] Input: flyer data, user information, emotion information

[0986] Data processing: Using algorithms, recipe information stored in the database is filtered based on evaluation criteria (ease of use, nutritional balance, cooking time). Based on emotional information, for example, recipes with short cooking times are prioritized for tired users.

[0987] Output: Optimal meal plan

[0988] Server: Pass the generated menu plan to the next recipe generation step.

[0989] Step 5: Recipe suggestions

[0990] Server: Runs a script that generates detailed recipes based on optimal menu plans.

[0991] Input: Optimal meal plan

[0992] Data processing: Using the Pandas library, generate a specific list of ingredients (e.g., 200g chicken thighs, 1 / 2 head of cabbage, 1 tomato, salt, and pepper) and cooking instructions (e.g., roughly chop the cabbage. Cut the chicken thighs into bite-sized pieces. Steam the cabbage and chicken thighs).

[0993] Output: Detailed recipe

[0994] Server: Pass the generated recipe to the shopping list generation step.

[0995] Step 6: Create a shopping list

[0996] Server: Based on the recipe information, it creates a list of the necessary ingredients and creates a shopping list.

[0997] Input: Detailed recipe

[0998] Data processing: Match the data with the grocery list at home, filter out unnecessary items, and generate the final shopping list.

[0999] Output: Final shopping list

[1000] Server: Sends the generated shopping list to the device.

[1001] Step 7: Display to the user

[1002] Terminal: Displays the menu, recipes, and shopping list received from the server on the user interface.

[1003] Input: Menus, recipes, shopping lists

[1004] Data processing: Organize and display information in a format that is easy for users to view. If the user makes any corrections, the data is sent back to the server.

[1005] Output: Menu, recipes, and shopping list displayed to the user

[1006] User: Review the information provided and make corrections as needed.

[1007] (Application example 2)

[1008] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1009] In modern society, as user lifestyles become more diverse, systems that suggest optimal meals based on an individual's emotional and health states are becoming increasingly important. However, conventional systems often suggest menus based solely on the user's preferences and budget, without taking the user's emotional state into consideration, resulting in low user satisfaction and low utilization rates. Furthermore, for busy modern people, finding the optimal meal for their emotional and physical states can be a significant burden. Therefore, there is a need for a system that utilizes the user's emotional information to suggest optimal menus.

[1010] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for acquiring flyer data, a means for acquiring emotion information, a means for generating a menu based on user information, a means for generating recipes based on the generated menu, and a means for creating a shopping list based on the generated recipe. This makes it possible to propose an optimal menu that reflects the user's emotional state.

[1011] A "user terminal" is an electronic device used by a user, and includes smartphones, tablets, PCs, etc.

[1012] "Flyer data" is advertising material issued by supermarkets and stores, and includes product names, prices, discount information, images, and the like.

[1013] "Emotional information" is data on the user's emotional state based on facial expressions, tone of voice, and other biological information.

[1014] "User information" refers to personal profile information such as the user's preferences, allergy information, budget, etc.

[1015] A "server" is a central system that processes and stores information on a network and communicates with clients (user terminals).

[1016] A "menu" is a meal plan made up of a combination of multiple dishes and ingredients.

[1017] A "recipe" is a list of ingredients and specific steps for making a dish.

[1018] A "shopping list" is a list of ingredients and products needed based on a recipe.

[1019] A "system" is a collection of devices and programs in which multiple functional components work in conjunction with one another.

[1020] This invention is a system that utilizes user emotion information to propose optimal menus for food delivery. This system includes a user terminal, an emotion engine, a server, and a network for communication among these.

[1021] System configuration

[1022] The user terminal is an electronic device such as a smartphone, tablet, or PC through which the user accesses the system. The user terminal has an internet connection function and allows input of the emotion engine and user information.

[1023] The emotion engine is software or hardware that analyzes a user's facial expressions, tone of voice, and other biometric information to generate emotional information about the user. The emotion engine is either built into the user terminal or operates as an external device connected to the user terminal.

[1024] The server is a central system that processes and stores emotion information, flyer data, and user information over the network. The server manages data such as user profiles, flyer databases, and recipe databases, and generates optimal menus and recipes for users.

[1025] System Operation

[1026] The operation of the system is as follows.

[1027] 1. Obtain flyer data:

[1028] The user's device downloads the latest menu information for the specified food delivery service and transmits it to the server, including product names, prices, images, etc.

[1029] 2. Acquiring emotional information:

[1030] The user device uses an emotion engine to obtain emotional information (e.g., "I'm tired" or "I'm stressed") from the user's facial expressions and tone of voice, and sends it to the server.

[1031] 3. Obtaining user information:

[1032] The server retrieves personal profile information such as the user's preferences, allergy information, budget, etc. from the database. For example, it confirms that the user's preference is Japanese food, that they have a dairy allergy, and that their weekly food budget is 5,000 yen.

[1033] 4. Menu generation:

[1034] The server generates a menu tailored to the user based on the flyer data, emotional information, and user information it has acquired. For example, if the user is tired, it will prioritize and suggest simple menus that require less time to cook.

[1035] 5. Recipe generation and shopping list creation:

[1036] Based on the generated menu, the server generates a detailed recipe and a shopping list for its execution, including a specific list of ingredients and cooking instructions.

[1037] 6. User Visibility:

[1038] The user terminal displays the menu, recipes, and shopping list sent from the server, and the user can check them and place orders as necessary.

[1039] Specific examples

[1040] Suppose a user is browsing a food delivery menu on a weekday evening. The user's device downloads the latest menu information (product name, price, and image), and the emotion engine recognizes the user's emotion, "tired," from their facial expression and tone of voice. The server takes this emotional information, along with the user's preferences and allergies, into consideration to suggest a menu that is easy to prepare and nutritious. For example, it suggests a dish called "chicken steamed with cabbage," and a detailed recipe and list of ingredients are displayed on the user's device.

[1041] Prompt Sentence Examples

[1042] "The user is extremely tired. Please suggest a menu that you would recommend at this time."

[1043] "Emotional information: I'm tired. Please suggest a menu that requires little cooking time and is within a budget of 5,000 yen."

[1044] In this way, it becomes possible to propose an optimal menu that reflects the user's emotional state.

[1045] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1046] Step 1:

[1047] The user device obtains the latest menu information from a specified food delivery service. Specifically, the user device accesses the food delivery service's API via the Internet and downloads menu information (product name, price, image, etc.). This menu information is sent from the user device to a server for use in subsequent processing steps. The input is the menu information from the food delivery service API, and the output is the menu information sent to the server.

[1048] Step 2:

[1049] The user device uses an emotion engine to obtain the user's emotional information. Specifically, the emotion engine analyzes the user's facial expression and tone of voice in real time and generates emotional information (for example, "tired" or "stressed"). This emotional information is sent from the user device to the server. The input is the user's facial expression and tone of voice, and the output is the emotional information sent to the server.

[1050] Step 3:

[1051] The server retrieves the user's personal profile information from a database, including preferences, allergies, budget, etc. This information is then combined with other data to generate an optimal menu. The input is a query to the server's database, and the output is the retrieved user information.

[1052] Step 4:

[1053] The server generates a menu suitable for the user based on the acquired flyer data, emotional information, and user information. Specifically, it prioritizes menus with short cooking times and menus with a relaxing effect based on emotional information (e.g., "tired"). The inputs are flyer data, emotional information, and user information, and the output is the generated menu.

[1054] Step 5:

[1055] The server generates a detailed recipe and a shopping list based on the generated menu. Specifically, the recipe includes a list of specific ingredients (e.g., 200g chicken thighs, 1 / 2 head of cabbage, 1 tomato) and cooking instructions (e.g., 1. Roughly chop the cabbage. 2. Cut the chicken thighs into bite-sized pieces), and the shopping list lists the ingredients needed based on the recipe. The input is the generated menu, and the output is a detailed recipe and a shopping list.

[1056] Step 6:

[1057] The user terminal displays the menu, recipes, and shopping list sent from the server. Specifically, the menu, recipes, and shopping list are displayed on the screen of the user terminal, and the user checks them and places an order as necessary. The system also encourages the user to select a menu based on their emotions when ordering. The input is the data sent from the server, and the output is the menu, recipes, and shopping list displayed on the user terminal.

[1058] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1059] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1060] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1061] [Fourth embodiment]

[1062] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1063] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1064] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1065] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1066] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1067] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1068] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1069] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1070] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1071] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1072] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1073] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1074] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1075] This invention is a system in which a server generates and proposes menus to users based on flyer data and user information acquired by the user terminal. This system helps users to shop efficiently and plan and implement appropriate menus and recipes even in their busy daily lives.

[1076] Program processing

[1077] This system is divided into the following main processing steps: Specific operations at each step are as follows:

[1078] 1. Obtain flyer data

[1079] A user uses a device to download the latest flyer data for a specified supermarket. For example, the user selects a flyer for Supermarket X. The device accesses Supermarket X's online service, obtains data such as product name, price, discount information, and images, and sends this data to the server.

[1080] 2. Obtaining user information

[1081] The server retrieves the user's personal profile information from the database. For example, it confirms that User A's preference is Japanese food, that he / she has a dairy allergy, and that his / her weekly food budget is 5,000 yen.

[1082] 3. Menu generation

[1083] The server generates a menu tailored to the user based on the flyer data and user information it has acquired. For example, it selects chicken thighs, cabbage, and tomatoes from the flyer data, extracts the user's favorite Japanese recipe from a home cooking database, and suggests "chicken steamed with cabbage."

[1084] 4. Recipe suggestions

[1085] The server generates a detailed recipe based on the menu, including a list of specific ingredients (e.g., 200g chicken thighs, 1 / 2 head of cabbage, 1 tomato, salt, and pepper) and cooking instructions (e.g., 1. Roughly shred the cabbage. 2. Cut the chicken thighs into bite-sized pieces. 3. Steam the cabbage and chicken thighs).

[1086] 5. Create a shopping list

[1087] The server creates a shopping list based on recipe information. For example, the list might include "400g chicken thighs, 1 head of cabbage, and 2 tomatoes." The server compares the list with the ingredients available at home and removes unnecessary items.

[1088] 6. User Display

[1089] The device displays the created menu, recipes, and shopping list to the user. The user can review this information and make corrections as necessary. For example, the user may decide that they do not need to buy cabbage because they already have it at home and correct the list accordingly.

[1090] Specific examples

[1091] For example, suppose a user wants to make a simple, nutritious meal for a weekday dinner. The user downloads a flyer for Supermarket X from their device and checks the specials it contains. They learn that chicken thighs, cabbage, and tomatoes are on sale, and use the system to suggest a meal. The server considers the user's preferences and allergy information and suggests "steamed chicken with cabbage." A detailed recipe is generated, and a shopping list is also created. The user can review this information on their device, make any necessary corrections, and then go shopping. In this way, users can efficiently plan their meals and ensure the shopping and cooking processes proceed smoothly.

[1092] The processing flow will be explained below.

[1093] Step 1:

[1094] The user makes a request on the terminal to download the latest flyer data of the specified supermarket.

[1095] Step 2:

[1096] The device accesses the supermarket's online service and retrieves the latest flyer data (product name, price, discount information, images, etc.).

[1097] Step 3:

[1098] The terminal transmits the acquired flyer data to the server.

[1099] Step 4:

[1100] The server retrieves the user's personal profile information (preferences, allergy information, budget, etc.) from a database.

[1101] Step 5:

[1102] The server generates a menu tailored to the user based on the acquired flyer data and user information.

[1103] Step 6:

[1104] The server retrieves recipes using the selected ingredients from a home cooking database and generates a detailed recipe, including a specific ingredient list and cooking instructions.

[1105] Step 7:

[1106] The server creates a shopping list of the necessary ingredients based on the generated recipe.

[1107] Step 8:

[1108] The server sends the shopping list to the terminal.

[1109] Step 9:

[1110] The device displays the created menu, recipes, and shopping list to the user, who can review and modify them as necessary.

[1111] Step 10:

[1112] The user goes shopping at the supermarket based on the shopping list and cooks using the purchased ingredients according to the recipe.

[1113] Example 1

[1114] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1115] Modern users face the challenge of efficiently shopping and planning and implementing appropriate menus and recipes in their busy daily lives. Conventional methods require users to collect market information and manually plan menus tailored to their individual needs, which takes a lot of time and effort. For this reason, there is a demand for a system that allows users to easily plan and implement nutritionally balanced meals while efficiently purchasing ingredients.

[1116] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1117] In this invention, the server includes means for generating a meal plan based on market information and user information, means for generating cooking instructions based on the generated meal plan, and means for creating a shopping list based on the generated cooking instructions. This allows users to automatically receive market information and meal plans and recipes tailored to their individual needs, allowing them to shop efficiently and easily plan and carry out appropriate meals.

[1118] A "user terminal" is an electronic device that allows a user to input and receive information.

[1119] "Market information" is data on products sold in specific markets or stores, and includes product names, prices, discount information, images, and the like.

[1120] "User information" is data about an individual user, including preferences, allergy information, budget, and the like.

[1121] A "meal plan" is a menu created based on the user's market information and user information.

[1122] "Cooking Instructions" are detailed instructions and ingredient lists for dishes based on a meal plan.

[1123] A "shopping list" is a list of ingredients and items needed based on the cooking procedure.

[1124] A "system" is a collection of hardware and software, including user terminals and servers, for realizing specific functions.

[1125] A "server" is a centralized computer system that processes and stores data.

[1126] This invention is a system that generates meal plans based on market information and user information and proposes them to users. This system helps users shop efficiently and plan and implement appropriate menus and recipes.

[1127] The system consists of user devices, servers, and supporting hardware and software. Specifically, it includes user devices such as smartphones and tablets, a server with a database, and an internet connection. The software supporting this hardware includes a market information acquisition API, a user profile management system, data analysis algorithms, machine learning models, and a user interface application.

[1128] Users can operate the application using a smartphone or tablet to obtain the latest information on a specific market. Market information includes product names, prices, discount information, images, etc. This information is sent to the server through a market information acquisition API. The server then retrieves personal profile information such as the user's preferences, allergies, and budget from a database.

[1129] The server generates an optimal meal plan for the user based on the acquired market information and user information. Data analysis algorithms and machine learning models are used to generate the meal plan. Specifically, specific ingredients are selected from market information, and appropriate recipes are extracted based on the user's preferences and allergy information. Through this series of processes, a specific recipe, such as "chicken steamed with cabbage," is suggested.

[1130] The server then creates a shopping list based on the generated recipes and ingredients list, which is then compared with the existing household inventory to eliminate unnecessary items, enabling efficient shopping.

[1131] The user terminal displays the meal plan, cooking instructions, and shopping list received from the server on a user interface. The user can review the information and make corrections as needed. For example, if there is already cabbage at home, the user can remove it from the shopping list.

[1132] A concrete example of this system implementation is when a user wants to make a simple and nutritious meal for a weekday dinner. The user retrieves flyer data for a specific market from their device, checks discounted products, and enters them into the system. The server takes into account the user's preferences and allergy information and suggests, for example, "chicken steamed with cabbage." A detailed recipe and shopping list are generated and displayed on the user interface.

[1133] As an example of a prompt sentence, the following sentence could be fed into a generative AI model:

[1134] "Please explain the system in which a user retrieves flyer data for a specific market from a device, and a server generates and proposes a meal plan to the user based on the user's preferences and allergy information. Specifically, please describe in detail each step from retrieving flyer data to proposing recipes and creating a shopping list."

[1135] In this way, by utilizing generative AI models, it is possible to provide users with efficient and nutritionally balanced meal plans.

[1136] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1137] Step 1:

[1138] The user uses a device to obtain market information. Specifically, the user selects a market, and the device obtains the latest market data through the market information acquisition API. The input here is the market information selected by the user, and the output is market data such as product name, price, discount information, and images. The device temporarily stores this data and sends it to the server.

[1139] Step 2:

[1140] The server retrieves user information. Based on the user ID entered when the user logged in to the application, the server uses an SQL query to retrieve the user's personal profile information from the database. The input is the user ID, and the output is the user's personal profile information, such as preferences, allergies, and budget. This information is stored in the user object within the program.

[1141] Step 3:

[1142] The server generates a meal plan based on market information and user information. The server uses data analysis algorithms and machine learning models to analyze the input market data and user information. The input is market data and user information, and the output is a meal plan suitable for the user (e.g., "chicken steamed with cabbage"). The server extracts related recipes from a home cooking database to suggest more detailed recipes.

[1143] Step 4:

[1144] The server generates detailed cooking instructions based on the meal plan. Specifically, it writes out the ingredient list and cooking instructions from the database and stores the data in a packet in JSON format. The input is the meal plan content, and the output is the ingredient list and detailed cooking instructions (e.g., "200g chicken thighs, 1 / 2 head of cabbage, 1 tomato, salt, pepper" and specific cooking instructions such as "Roughly chop the cabbage. Cut the chicken thighs into bite-sized pieces. Steam the cabbage and chicken thighs.").

[1145] Step 5:

[1146] The server creates a shopping list based on the cooking instructions. The input is the cooking instructions and the household inventory list, and the output is a shopping list with unnecessary items removed. The server compares the existing household inventory list with the newly created ingredient list and removes unnecessary items (e.g., if cabbage is already in the household, remove it from the shopping list).

[1147] Step 6:

[1148] The device displays the created meal plan, cooking instructions, and shopping list to the user. The input is the data received from the server, and the output is the information displayed on the user interface. The device application retrieves this latest data and acts as an interface for the user to review and modify. The user can then review the displayed information, modify it as necessary, and go shopping.

[1149] (Application example 1)

[1150] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1151] Conventional shopping planning support systems have the problem that it takes a lot of time and effort for users to efficiently shop and plan and implement appropriate menus and recipes in their busy daily lives. In particular, the complicated process of generating menus based on the latest flyer information and then ordering the ingredients through a delivery service is a challenge. In addition, the system's inability to provide optimal menu suggestions that meet the user's specific needs can lead to low user satisfaction.

[1152] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1153] In this invention, the server includes means for generating a menu based on flyer data and user information, means for generating recipes based on the generated menu, means for creating a shopping list based on the generated recipe, and means for the terminal to generate prompt sentences using a generative AI model to suggest an optimal menu in response to user requests, and means for providing a delivery service based on the generated shopping list. This enables users to efficiently plan their shopping, smoothly obtain the necessary ingredients through a delivery service, and implement an optimal menu.

[1154] A "user terminal" is a communication device operated by a user, and includes a smartphone, a tablet terminal, a personal computer, and the like.

[1155] "Flyer data" is product information provided by a sales store, and includes data such as product name, price, discount information, and images.

[1156] "User information" is information that indicates the individual characteristics and preferences of a user, including food preferences, allergy information, budget, and the like.

[1157] A "server" is a computer system that processes large amounts of data, and receives and processes data from user terminals.

[1158] The "menu creation means" has the function of creating a specific meal plan based on the acquired flyer data and user information.

[1159] The "recipe generation means" has a function of creating specific cooking procedures and a list of ingredients based on the generated menu.

[1160] The "shopping list creation means" has a function of creating a list of ingredients needed based on the generated recipe.

[1161] The "delivery service providing means" realizes the function of ordering and delivering the necessary ingredients based on the generated shopping list.

[1162] A "generative AI model" is an artificial intelligence model that uses machine learning algorithms to suggest optimal menus based on user requests.

[1163] A "prompt sentence" is a sentence input into a generative AI model, and is text that describes the user's requests and conditions.

[1164] The present invention provides a system that combines a user terminal, a server, and a delivery service to efficiently plan shopping and propose menus. Specific embodiments of the system will be described in detail below.

[1165] 1. Hardware Configuration

[1166] In this embodiment, the following hardware is used.

[1167] User devices: smartphones, tablets, personal computers, etc.

[1168] Server: A computer system that processes large amounts of data

[1169] Delivery service: online ordering and delivery system

[1170] 2. Software Configuration

[1171] The system uses the following software:

[1172] Python: Used to develop programs

[1173] Flask: a web framework

[1174] The Requests library: Executing web requests

[1175] SQLite or Firebase: a database for storing user information

[1176] Generative AI model: An artificial intelligence model for proposing optimal menus based on user requests

[1177] 3. Data processing and calculations

[1178] The main data processing and calculation flow performed by the system is shown below.

[1179] (1) Obtaining flyer data

[1180] Users can access the latest flyer data for a designated store from their smartphone or other device. This is done by accessing the store's online service and downloading data such as product name, price, discount information, and images. This information is then sent to a server via the user's device.

[1181] (2) Acquisition of user information

[1182] The server retrieves the user's profile information from a database, including the user's food preferences, allergy information, budget, etc.

[1183] (3) Menu generation

[1184] The server generates the optimal menu based on the acquired flyer data and user information. For example, it selects special offers on chicken thighs, cabbage, tomatoes, etc. from the flyer data and suggests "steamed chicken with cabbage" taking into account the user's preferences and allergies.

[1185] (4) Recipe suggestions

[1186] Once the menu is decided, the server generates a detailed recipe, including a list of specific ingredients (e.g., 200g chicken thighs, 1 / 2 head of cabbage, 1 tomato, salt, and pepper) and cooking instructions (e.g., 1. Roughly shred the cabbage. 2. Cut the chicken thighs into bite-sized pieces. 3. Steam the cabbage and chicken thighs).

[1187] (5) Creating a shopping list

[1188] The server creates a shopping list based on the generated recipe. For example, the list might include "400g of chicken thighs, 1 head of cabbage, and 2 tomatoes." It then compares the list with ingredients already in the home and eliminates any duplicates.

[1189] (6) Providing delivery services

[1190] Implement a means to order the necessary ingredients from a delivery service based on the generated shopping list. This includes the process of ordering ingredients from a user terminal through a delivery service and having them delivered to a specified location.

[1191] 4. Use of generative AI models

[1192] The server uses a generative AI model to propose the optimal menu based on the user's request. Specifically, the AI ​​generates an appropriate menu based on the prompt text entered by the user.

[1193] Prompt Sentence Examples

[1194] "Please suggest a balanced dinner menu based on the user's preferences and the latest flyer information. It should be Japanese, dairy-free, and within a budget of 5,000 yen."

[1195] In this way, the present invention enables users to efficiently plan their shopping, smoothly obtain the necessary ingredients through a delivery service, and implement an optimal menu.

[1196] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1197] Step 1:

[1198] The user terminal acquires the latest flyer data of the designated store.

[1199] Input: User-specified store identification information.

[1200] Operation: The user's device accesses the online service and downloads flyer data such as product names, prices, discount information, and images.

[1201] Output: The retrieved flyer data.

[1202] Step 2:

[1203] The server retrieves the user information from the database.

[1204] Input: User ID.

[1205] How it works: The server accesses a local or cloud database (e.g., SQLite, Firebase) to retrieve information about the user, such as their food preferences, allergies, and budget.

[1206] Output: User information.

[1207] Step 3:

[1208] The server generates the optimal menu based on flyer data and user information.

[1209] Input: Flyer data, user information.

[1210] Operation: The server selects special sale information from flyer data and creates a menu taking into account the user's preferences and allergy information.

[1211] Output: The generated menu.

[1212] Step 4:

[1213] The server generates detailed recipes based on the generated menu.

[1214] Input: The generated menu.

[1215] How it works: The server extracts suitable recipes from a home cooking database and generates specific ingredient lists and cooking instructions.

[1216] Output: A detailed recipe.

[1217] Step 5:

[1218] The server creates a shopping list based on the generated recipe.

[1219] Input: A detailed recipe.

[1220] How it works: The server creates a list of ingredients needed based on the recipe, compares it with the list of ingredients already in the house, and weeds out any duplicates.

[1221] Output: Shopping list.

[1222] Step 6:

[1223] The server provides delivery services based on the shopping list.

[1224] Enter: shopping list.

[1225] How it works: The server uses the delivery service's API to order ingredients and arrange for delivery to the specified location.

[1226] Output: Delivery order confirmation information.

[1227] Step 7:

[1228] The device uses a generative AI model to generate prompts that suggest the optimal menu based on the user's requests.

[1229] Input: User request (e.g., "Please suggest a balanced dinner menu based on the user's preferences and the latest flyer information. Japanese food, dairy-free, within a budget of 5,000 yen.").

[1230] How it works: Enter a prompt into the generative AI model, and the AI ​​will generate an appropriate menu.

[1231] Output: The optimal menu suggested to the user.

[1232] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1233] This invention is a menu suggestion and recipe provision system that combines an emotion engine that recognizes the user's emotions. This system makes the user's eating habits more efficient by having the server generate optimal menus, recipes, and shopping lists based on flyer data, user information, and emotion information acquired by the user's terminal.

[1234] Program processing

[1235] This system is divided into the following main processing steps: Specific operations at each step are as follows:

[1236] 1. Obtain flyer data

[1237] The user requests the download of the latest flyer data for the specified supermarket on their device. The device accesses the supermarket's online service and obtains data such as product names, prices, discount information, and images. The obtained flyer data is then sent to the server.

[1238] 2. Obtaining user information

[1239] The server retrieves the user's personal profile information (preferences, allergy information, budget, etc.) from the database. For example, it confirms that User A's preference is Japanese food, that he has a dairy allergy, and that his weekly food budget is 5,000 yen.

[1240] 3. Acquiring emotional information

[1241] Using an emotion engine built into or connected to the user's device, the system acquires emotional information from the user's facial expressions and tone of voice while they are browsing the flyer. For example, it can recognize emotions such as "tired" or "stressed."

[1242] 4. Menu generation

[1243] The server generates a menu suited to the user based on the acquired flyer data, user information, and emotional information. For example, if the user is tired, it will prioritize simple recipes with short cooking times. It will also select chicken thighs, cabbage, and tomatoes from the flyer data and suggest "chicken steamed with cabbage."

[1244] 5. Recipe suggestions

[1245] The server generates a detailed recipe based on the generated menu, including a list of specific ingredients (e.g., 200g chicken thighs, 1 / 2 head of cabbage, 1 tomato, salt, and pepper) and cooking instructions (e.g., 1. Roughly shred the cabbage. 2. Cut the chicken thighs into bite-sized pieces. 3. Steam the cabbage and chicken thighs).

[1246] 6. Create a shopping list

[1247] The server creates a shopping list based on recipe information. For example, the list might include "400g chicken thighs, 1 head of cabbage, and 2 tomatoes." The server compares the list with the ingredients available at home and removes unnecessary items.

[1248] 7. User Display

[1249] The device displays the meal plan, recipes, and shopping list to the user, who can review and modify them as needed. For example, the device may adjust the display to present particularly simple steps for a tired user.

[1250] Specific examples

[1251] For example, suppose a user finds a sale flyer at a supermarket on a weekday evening and needs to prepare dinner. The user downloads the supermarket flyer data using their device and checks the sale items included in the flyer. The emotion engine recognizes the emotion "tired" from the user's facial expression and tone of voice. The server takes into account the user's preferences and allergy information, as well as the user's emotional fatigue information, and suggests "steamed chicken with cabbage," which is easy to prepare and has excellent nutritional value, as a menu item. A detailed recipe and shopping list are generated and displayed on the user's device. The user can review the list, make any necessary edits, then go shopping at the supermarket and easily prepare the dish at home.

[1252] In this way, the present invention reflects the user's emotional state and provides appropriate menus and recipes, thereby reducing the time and effort required for cooking and easing the burden on the user.

[1253] The processing flow will be explained below.

[1254] Step 1:

[1255] The user makes a request on the terminal to download the latest flyer data of the specified supermarket.

[1256] Step 2:

[1257] The device accesses the online service of the specified supermarket and obtains data such as product name, price, discount information, images, etc. The obtained flyer data is then sent to the server.

[1258] Step 3:

[1259] The server retrieves the user's personal profile information (preferences, allergy information, budget, etc.) from a database.

[1260] Step 4:

[1261] Using an emotion engine built into or connected to the user's device, the system acquires emotional information from the user's facial expressions and tone of voice while they are browsing the flyer. For example, it can recognize emotions such as "tired" or "stressed."

[1262] Step 5:

[1263] The emotion information acquired by the user terminal is transmitted to the server.

[1264] Step 6:

[1265] The server generates a menu suited to the user based on the acquired flyer data, user information, and emotional information. For example, if the user is tired, it will prioritize simple recipes with short cooking times.

[1266] Step 7:

[1267] The server retrieves recipes using the selected ingredients from a home cooking database and generates a detailed recipe, including a specific ingredient list and cooking instructions.

[1268] Step 8:

[1269] The server creates a shopping list based on the generated recipe. For example, the list might include "400g of chicken thighs, 1 head of cabbage, and 2 tomatoes." The server compares the list with the ingredients available at home and removes unnecessary items.

[1270] Step 9:

[1271] The server sends the shopping list to the terminal.

[1272] Step 10:

[1273] The device displays the created menu, recipes, and shopping list to the user, who can review them and make any necessary corrections.

[1274] Step 11:

[1275] The user goes shopping at the supermarket based on the shopping list and cooks food according to the recipe using the purchased ingredients.

[1276] Example 2

[1277] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1278] In recent years, for many users who lead busy lives, preparing meals every day requires time and effort. In particular, it is difficult to propose menus that take into account the user's emotional state and preferences. Therefore, there is a need for menu suggestions that are appropriate for users who are tired or have specific preferences or allergies. There is also a need for support in creating efficient shopping lists and cooking based on those lists. There is a need for systems and methods that can solve these problems.

[1279] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for generating a menu based on flyer data, user information, and emotion information, a means for generating recipes based on the generated menu, and a means for creating a shopping list based on the generated recipe. This makes it possible to provide an optimal menu according to the user's emotions and individual needs, and to save the effort of creating a shopping list and cooking.

[1280] "User terminal" refers to an electronic device such as a computer device or smartphone that is directly operated by a user.

[1281] "Flyer data" is data containing information on the latest products offered by retailers such as supermarkets, and includes product names, prices, discount information, images, and the like.

[1282] "User information" refers to information about a user's personal profile, including preferences, allergy information, budget, and the like.

[1283] "Emotional information" refers to data that indicates the emotional state of the user, such as facial expressions and tone of voice.

[1284] "Menu" refers to the combination of dishes and menus suggested to optimize the user's diet.

[1285] "Recipe" refers to a document or information that includes a specific list of ingredients and cooking instructions based on a menu.

[1286] A "shopping list" is a list of ingredients needed to prepare a recipe, and is intended to support efficient shopping.

[1287] A "server" refers to a computer system that processes requests from terminals via a network and sends and receives necessary data.

[1288] This invention is a menu suggestion and recipe provision system that combines an emotion engine that recognizes the user's emotions. Based on flyer data, user information, and emotion information acquired by the user's terminal, the server generates optimal menus, recipes, and shopping lists and displays them to the user, thereby streamlining the user's eating habits.

[1289] Hardware and software used

[1290] User device: An electronic device that can be directly operated by a user, such as a computer or smartphone, to which a camera, microphone, etc. are connected.

[1291] Server: A computer system that processes requests from terminals and sends and receives data over a network.

[1292] Emotion engine: Software that recognizes emotions from a user's facial expressions and tone of voice. For example, it uses machine learning libraries such as TensorFlow.

[1293] Database: A database system, such as MySQL, for storing user information and flyer data.

[1294] Programming languages ​​and libraries: Python, Pandas, HTTP requests, etc.

[1295] Specific operation explanation

[1296] Obtaining flyer data

[1297] When a user downloads flyer information for a particular supermarket on their device, the device accesses the online service using an HTTP request to obtain data such as product names, prices, discount information, images, etc. This data is temporarily stored in local storage and then sent to the server.

[1298] Obtaining user information

[1299] The server queries the database based on the user ID and obtains information such as the user's preferences, allergies, budget, etc. The data is loaded into memory and handed over to the next process.

[1300] Acquiring emotional information

[1301] The user device uses a camera and microphone to collect facial expressions and tone of voice, which are then analyzed by the emotion engine. TensorFlow is used for emotion recognition, and the estimated results are sent to the server.

[1302] Menu and recipe generation

[1303] The server runs an algorithm to generate optimal menus based on collected flyer data, user information, and emotional information. For example, if the server detects that the user is tired, it prioritizes recipes with short cooking times. This is done using Python scripts and the Pandas library. From the generated menu, a detailed recipe is created, including a specific ingredient list and cooking instructions.

[1304] Creating a shopping list

[1305] Based on the generated recipe, the necessary ingredients are compiled into a shopping list, which is then compared with the ingredients in your home and unnecessary items are removed to create the final shopping list.

[1306] What the user sees

[1307] Finally, the generated menu, detailed recipes, and shopping list are displayed on the user's device, where the user can review the information provided and make any necessary corrections.

[1308] Prompt Sentence Examples

[1309] For example, an example of a prompt sentence to be input into a generative AI model is, "When a user finds a sale flyer at the supermarket on a weekday evening and needs to prepare dinner, please explain the process of the system recognizing the user's tiredness from their facial expression and tone of voice, and suggesting a menu that is easy to prepare while taking those emotions into consideration."

[1310] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1311] Step 1: Obtain flyer data

[1312] User: The user operates the device and issues a request to download flyer data for a specific supermarket.

[1313] Input: Supermarket flyer URL

[1314] Terminal: Upon receiving a request from the user, the terminal uses an HTTP request to access the supermarket's online service and retrieve flyer data.

[1315] Data processing: The acquired data includes product names, prices, discount information, images, etc. This data is parsed, organized, and saved.

[1316] Output: Organized flyer data

[1317] Device: The organized flyer data is temporarily stored in local storage, and then sent to the server via an HTTP POST request.

[1318] Step 2: Get user information

[1319] Server: Queries the database based on the user ID and retrieves the user's personal profile information.

[1320] Input: User ID

[1321] Data processing: Information retrieved from the database includes user preferences, allergies, budget, etc. This information is loaded into memory and organized for further processing.

[1322] Output: User's personal profile information

[1323] Server: Records the acquired user information in a log and passes it to the subsequent menu generation process.

[1324] Step 3: Acquiring emotional information

[1325] Device: Uses a camera and microphone to capture your facial expressions and tone of voice.

[1326] Input: User facial and voice data

[1327] Data processing: Using an emotion recognition engine (e.g., TensorFlow), the collected data is analyzed to estimate the user's emotional state, and emotional information such as fatigue and stress levels is obtained.

[1328] Output: User's emotional information

[1329] Device: Sends analyzed emotion information to the server.

[1330] Step 4: Create a menu

[1331] Server: Runs an algorithm to generate a menu suitable for the user based on the acquired flyer data, user information, and emotional information.

[1332] Input: flyer data, user information, emotion information

[1333] Data processing: Using algorithms, recipe information stored in the database is filtered based on evaluation criteria (ease of use, nutritional balance, cooking time). Based on emotional information, for example, recipes with short cooking times are prioritized for tired users.

[1334] Output: Optimal meal plan

[1335] Server: Pass the generated menu plan to the next recipe generation step.

[1336] Step 5: Recipe suggestions

[1337] Server: Runs a script that generates detailed recipes based on optimal menu plans.

[1338] Input: Optimal meal plan

[1339] Data processing: Using the Pandas library, generate a specific list of ingredients (e.g., 200g chicken thighs, 1 / 2 head of cabbage, 1 tomato, salt, and pepper) and cooking instructions (e.g., roughly chop the cabbage. Cut the chicken thighs into bite-sized pieces. Steam the cabbage and chicken thighs).

[1340] Output: Detailed recipe

[1341] Server: Pass the generated recipe to the shopping list generation step.

[1342] Step 6: Create a shopping list

[1343] Server: Based on the recipe information, it creates a list of the necessary ingredients and creates a shopping list.

[1344] Input: Detailed recipe

[1345] Data processing: Match the data with the grocery list at home, filter out unnecessary items, and generate the final shopping list.

[1346] Output: Final shopping list

[1347] Server: Sends the generated shopping list to the device.

[1348] Step 7: Display to the user

[1349] Terminal: Displays the menu, recipes, and shopping list received from the server on the user interface.

[1350] Input: Menus, recipes, shopping lists

[1351] Data processing: Organize and display information in a format that is easy for users to view. If the user makes any corrections, the data is sent back to the server.

[1352] Output: Menu, recipes, and shopping list displayed to the user

[1353] User: Review the information provided and make corrections as needed.

[1354] (Application example 2)

[1355] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1356] In modern society, as user lifestyles become more diverse, systems that suggest optimal meals based on an individual's emotional and health states are becoming increasingly important. However, conventional systems often suggest menus based solely on the user's preferences and budget, without taking the user's emotional state into consideration, resulting in low user satisfaction and low utilization rates. Furthermore, for busy modern people, finding the optimal meal for their emotional and physical states can be a significant burden. Therefore, there is a need for a system that utilizes the user's emotional information to suggest optimal menus.

[1357] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for acquiring flyer data, a means for acquiring emotion information, a means for generating a menu based on user information, a means for generating recipes based on the generated menu, and a means for creating a shopping list based on the generated recipe. This makes it possible to propose an optimal menu that reflects the user's emotional state.

[1358] A "user terminal" is an electronic device used by a user, and includes smartphones, tablets, PCs, etc.

[1359] "Flyer data" is advertising material issued by supermarkets and stores, and includes product names, prices, discount information, images, and the like.

[1360] "Emotional information" is data on the user's emotional state based on facial expressions, tone of voice, and other biological information.

[1361] "User information" refers to personal profile information such as the user's preferences, allergy information, budget, etc.

[1362] A "server" is a central system that processes and stores information on a network and communicates with clients (user terminals).

[1363] A "menu" is a meal plan made up of a combination of multiple dishes and ingredients.

[1364] A "recipe" is a list of ingredients and specific steps for making a dish.

[1365] A "shopping list" is a list of ingredients and products needed based on a recipe.

[1366] A "system" is a collection of devices and programs in which multiple functional components work in conjunction with one another.

[1367] This invention is a system that utilizes user emotion information to propose optimal menus for food delivery. This system includes a user terminal, an emotion engine, a server, and a network for communication among these.

[1368] System configuration

[1369] The user terminal is an electronic device such as a smartphone, tablet, or PC through which the user accesses the system. The user terminal has an internet connection function and allows input of the emotion engine and user information.

[1370] The emotion engine is software or hardware that analyzes a user's facial expressions, tone of voice, and other biometric information to generate emotional information about the user. The emotion engine is either built into the user terminal or operates as an external device connected to the user terminal.

[1371] The server is a central system that processes and stores emotion information, flyer data, and user information over the network. The server manages data such as user profiles, flyer databases, and recipe databases, and generates optimal menus and recipes for users.

[1372] System Operation

[1373] The operation of the system is as follows.

[1374] 1. Obtain flyer data:

[1375] The user's device downloads the latest menu information for the specified food delivery service and transmits it to the server, including product names, prices, images, etc.

[1376] 2. Acquiring emotional information:

[1377] The user device uses an emotion engine to obtain emotional information (e.g., "I'm tired" or "I'm stressed") from the user's facial expressions and tone of voice, and sends it to the server.

[1378] 3. Obtaining user information:

[1379] The server retrieves personal profile information such as the user's preferences, allergy information, budget, etc. from the database. For example, it confirms that the user's preference is Japanese food, that they have a dairy allergy, and that their weekly food budget is 5,000 yen.

[1380] 4. Menu generation:

[1381] The server generates a menu tailored to the user based on the flyer data, emotional information, and user information it has acquired. For example, if the user is tired, it will prioritize and suggest simple menus that require less time to cook.

[1382] 5. Recipe generation and shopping list creation:

[1383] Based on the generated menu, the server generates a detailed recipe and a shopping list for its execution, including a specific list of ingredients and cooking instructions.

[1384] 6. User Visibility:

[1385] The user terminal displays the menu, recipes, and shopping list sent from the server, and the user can check them and place orders as necessary.

[1386] Specific examples

[1387] Suppose a user is browsing a food delivery menu on a weekday evening. The user's device downloads the latest menu information (product name, price, and image), and the emotion engine recognizes the user's emotion, "tired," from their facial expression and tone of voice. The server takes this emotional information, along with the user's preferences and allergies, into consideration to suggest a menu that is easy to prepare and nutritious. For example, it suggests a dish called "chicken steamed with cabbage," and a detailed recipe and list of ingredients are displayed on the user's device.

[1388] Prompt Sentence Examples

[1389] "The user is extremely tired. Please suggest a menu that you would recommend at this time."

[1390] "Emotional information: I'm tired. Please suggest a menu that requires little cooking time and is within a budget of 5,000 yen."

[1391] In this way, it becomes possible to propose an optimal menu that reflects the user's emotional state.

[1392] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1393] Step 1:

[1394] The user device obtains the latest menu information from a specified food delivery service. Specifically, the user device accesses the food delivery service's API via the Internet and downloads menu information (product name, price, image, etc.). This menu information is sent from the user device to a server for use in subsequent processing steps. The input is the menu information from the food delivery service API, and the output is the menu information sent to the server.

[1395] Step 2:

[1396] The user device uses an emotion engine to obtain the user's emotional information. Specifically, the emotion engine analyzes the user's facial expression and tone of voice in real time and generates emotional information (for example, "tired" or "stressed"). This emotional information is sent from the user device to the server. The input is the user's facial expression and tone of voice, and the output is the emotional information sent to the server.

[1397] Step 3:

[1398] The server retrieves the user's personal profile information from a database, including preferences, allergies, budget, etc. This information is then combined with other data to generate an optimal menu. The input is a query to the server's database, and the output is the retrieved user information.

[1399] Step 4:

[1400] The server generates a menu suitable for the user based on the acquired flyer data, emotional information, and user information. Specifically, it prioritizes menus with short cooking times and menus with a relaxing effect based on emotional information (e.g., "tired"). The inputs are flyer data, emotional information, and user information, and the output is the generated menu.

[1401] Step 5:

[1402] The server generates a detailed recipe and a shopping list based on the generated menu. Specifically, the recipe includes a list of specific ingredients (e.g., 200g chicken thighs, 1 / 2 head of cabbage, 1 tomato) and cooking instructions (e.g., 1. Roughly chop the cabbage. 2. Cut the chicken thighs into bite-sized pieces), and the shopping list lists the ingredients needed based on the recipe. The input is the generated menu, and the output is a detailed recipe and a shopping list.

[1403] Step 6:

[1404] The user terminal displays the menu, recipes, and shopping list sent from the server. Specifically, the menu, recipes, and shopping list are displayed on the screen of the user terminal, and the user checks them and places an order as necessary. The system also encourages the user to select a menu based on their emotions when ordering. The input is the data sent from the server, and the output is the menu, recipes, and shopping list displayed on the user terminal.

[1405] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1406] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1407] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1408] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1409] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1410] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1411] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1412] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1413] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1414] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1415] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1416] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1417] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1418] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1419] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1420] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1421] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1422] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1423] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1424] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1425] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1426] The following is further disclosed regarding the above embodiment.

[1427] (Claim 1)

[1428] A means for the user terminal to acquire flyer data;

[1429] A means for obtaining user information;

[1430] A means for the server to generate a menu based on flyer data and user information;

[1431] A server generates a recipe based on the generated menu;

[1432] a means for the server to create a shopping list based on the generated recipe;

[1433] a means for displaying the created menu, recipes, and shopping list to the user in the terminal;

[1434] A system including:

[1435] (Claim 2)

[1436] 2. The system according to claim 1, wherein the flyer data includes product names, prices, discount information, and images.

[1437] (Claim 3)

[1438] 2. The system of claim 1, wherein the user information includes preferences, allergy information, and budget.

[1439] "Example 1"

[1440] (Claim 1)

[1441] A means for acquiring market information in a user terminal;

[1442] A means for obtaining user information;

[1443] A server generates a meal plan based on market information and user information;

[1444] A means for the server to generate cooking instructions based on the generated meal plan;

[1445] A means for the server to create a shopping list based on the generated cooking instructions;

[1446] means for the terminal to display the created meal plan, cooking instructions, and shopping list to the user;

[1447] A system including:

[1448] (Claim 2)

[1449] 2. The system of claim 1, wherein the market information includes product names, prices, discount information, and images.

[1450] (Claim 3)

[1451] 2. The system of claim 1, wherein the user information includes preferences, allergy information, and budget.

[1452] "Application Example 1"

[1453] (Claim 1)

[1454] A means for the user terminal to acquire flyer data;

[1455] A means for obtaining user information;

[1456] A means for the server to generate a menu based on flyer data and user information;

[1457] A server generates a recipe based on the generated menu;

[1458] a means for the server to create a shopping list based on the generated recipe;

[1459] a means for displaying the created menu, recipes, and shopping list to the user in the terminal;

[1460] A means for the server to provide a delivery service based on the generated shopping list;

[1461] A system including:

[1462] (Claim 2)

[1463] 2. The system according to claim 1, wherein the flyer data includes product names, prices, discount information, and images.

[1464] (Claim 3)

[1465] 2. The system of claim 1, wherein the user information includes preferences, allergy information, and budget.

[1466] (Claim 4)

[1467] The system of claim 1, wherein the terminal includes means for generating prompt sentences using a generative AI model to suggest optimal menus according to user requests.

[1468] "Example 2: Combining Emotion Engines"

[1469] (Claim 1)

[1470] A means for the user terminal to acquire flyer data;

[1471] A means for acquiring emotion information of a user in a user terminal;

[1472] A server generates a menu based on flyer data, user information, and emotion information;

[1473] A server generates a recipe based on the generated menu;

[1474] a means for the server to create a shopping list based on the generated recipe;

[1475] a means for displaying the created menu, recipes, and shopping list to the user in the terminal;

[1476] A system including:

[1477] (Claim 2)

[1478] 2. The system according to claim 1, wherein the flyer data includes product names, prices, discount information, and images.

[1479] (Claim 3)

[1480] 2. The system of claim 1, wherein the user information includes preferences, allergy information, and budget.

[1481] "Application example 2 when combining emotion engines"

[1482] (Claim 1)

[1483] A means for the user terminal to acquire flyer data;

[1484] A means for acquiring emotion information in a user terminal;

[1485] A means for obtaining user information;

[1486] A server generates a menu based on flyer data, emotion information, and user information;

[1487] A server generates a recipe based on the generated menu;

[1488] a means for the server to create a shopping list based on the generated recipe;

[1489] a means for displaying the created menu, recipes, and shopping list to the user in the terminal;

[1490] A system including:

[1491] (Claim 2)

[1492] 2. The system according to claim 1, wherein the flyer data includes product names, prices, discount information, and images.

[1493] (Claim 3)

[1494] 2. The system of claim 1, wherein the user information includes preferences, allergy information, and budget.

[1495] (Claim 4)

[1496] 2. The system according to claim 1, further comprising means for suggesting a menu suited to the emotional state of the user based on the emotional information. [Explanation of symbols]

[1497] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for the user terminal to acquire flyer data; A means for obtaining user information; A means for the server to generate a menu based on flyer data and user information; A server generates a recipe based on the generated menu; a means for the server to create a shopping list based on the generated recipe; a means for displaying the created menu, recipes, and shopping list to the user in the terminal; A system including:

2. 2. The system according to claim 1, wherein the flyer data includes product names, prices, discount information, and images.

3. 2. The system of claim 1, wherein the user information includes preferences, allergy information, and budget.

Images