system

Abstract

We provide the system. [Solution] Means of obtaining the latest sale information, A means of obtaining allergy information and dietary conditions such as calorie restrictions entered by the user, A means for generating a menu based on the aforementioned special offer information and the aforementioned meal conditions, A means for creating a list of necessary ingredients based on the generated menu, A means of placing an order based on the aforementioned list of ingredients and delivering the ingredients, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the modern busy living environment, especially for people who cook their own meals, planning and actually executing an economical and healthy diet is a major challenge. Checking supermarket sale information every time and considering a menu taking into account allergies and calorie restrictions is time - consuming and often difficult due to time constraints. Also, this easily causes food cost waste and food loss. Furthermore, for people who have difficulty going shopping, such as the elderly and parents with children, it is even more difficult to prepare the necessary ingredients. There is a need for a system that comprehensively solves these problems.

Means for Solving the Problems

[0005] The present invention solves the aforementioned problems by providing a system that collects sale information, generates menus that take into account the user's dietary conditions, and delivers the necessary ingredients. Specifically, the invention provides a system that includes means for acquiring the latest sale information, means for acquiring allergy information and dietary conditions such as calorie restrictions entered by the user, means for generating menus based on the sale information and dietary conditions, means for creating a list of necessary ingredients based on the generated menu, and means for placing an order based on the ingredient list and delivering the ingredients. As a result, the user can obtain economical and healthy menus without any hassle, and have the necessary ingredients delivered to their door. Furthermore, by acquiring sale information from multiple partner retailers, it is possible to propose the optimal menu from a wider range of options.

[0006] "Special sale information" refers to information about products that are discounted for a limited time, obtained from multiple affiliated retailers.

[0007] "Allergy information" refers to information entered by the user about specific foods or ingredients that they or their family members are allergic to.

[0008] "Calorie restriction" refers to information that users enter regarding target values ​​or conditions for limiting the total calories they consume within a specific period.

[0009] A "menu" is a series of meal suggestions generated taking into account the user's allergy information, calorie restrictions, and special offer information.

[0010] A "food ingredient list" is a list that enumerates the types and quantities of ingredients needed based on the menu.

[0011] "Home delivery" is a service that delivers groceries to a specified address based on a list of ingredients selected by the user.

[0012] The "system" is a comprehensive software and hardware configuration consisting of a series of means for acquiring special offer information, collecting user information, generating menus, creating ingredient lists, and delivering ingredients. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0014] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0019] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] This invention is a system that targets people who cook at home and generates economical and healthy menus by taking into account supermarket sale information and dietary conditions specified by the user (allergy information and calorie restrictions). Furthermore, by linking with delivery services, it is possible to deliver ingredients to people who have difficulty going shopping, such as those raising children or the elderly.

[0035] This system mainly consists of the following elements:

[0036] 1. Gathering information on special offers

[0037] 2. Entering and saving user information

[0038] 3. Menu generation

[0039] 4. Providing a list of ingredients

[0040] 5. Use of delivery services

[0041] Program processing flow

[0042] Gathering information on special offers

[0043] The server uses a script that runs at specific times to retrieve special offer information from the APIs of multiple partner retailers. For example, the server sends a GET request to a specific URL at 3 AM every day and saves the received JSON data to the database. In this way, the latest special offer information is always stored in the database.

[0044] Entering user information

[0045] Users input dietary information such as allergy details and calorie restrictions through applications or websites. The device receives this information and sends it to the server. For example, a user might enter information such as "I have a nut allergy" and "I have a 1500 calorie limit per day" into an input form. The device then sends this data to the server as a POST request in JSON format.

[0046] User information storage

[0047] The server stores the received user information in a database. This makes it easy to reuse information that the user may need in the future.

[0048] Menu generation

[0049] The server uses a generative AI to generate menus based on user information and acquired sale information. For example, the server sends the aforementioned user information and sale information to the AI, which then suggests a menu like "chicken curry, salad, and rice."

[0050] Menu

[0051] The generated menu is sent from the server to the user's device and displayed visually on the device. Users can check the menu and its details on devices such as PCs and smartphones.

[0052] Providing a list of ingredients

[0053] Based on the menu, the server creates a list of necessary ingredients and sends it to the user's terminal. For example, an ingredient list such as "chicken, curry powder, various vegetables, rice" will be displayed.

[0054] Use of delivery service

[0055] When a user requests delivery, they make a selection through their device, and that information is sent to the server. The server then sends the necessary information to the partner delivery service, and the groceries are delivered to the specified address. For example, the server might send the "delivery address and list of required groceries" as a POST request to the delivery company's API.

[0056] Specific example

[0057] In reality, the following scenario is possible:

[0058] 1. Gathering information on special offers:

[0059] The server retrieves special offer information from "Store A" and "Store B," and items such as "chicken," "curry powder," and "vegetables" are listed as special offer products.

[0060] 2. Entering and saving user information:

[0061] The user accesses the website and enters "nut allergy" as allergy information and "1500 calories" as calorie restriction.

[0062] 3. Menu generation:

[0063] Based on the aforementioned special offer information and user information, the server uses a generative AI to generate a menu consisting of "chicken curry, salad, and rice."

[0064] 4. Menu provision:

[0065] The server sends this menu to the user's device, and the user checks the menu on their smartphone.

[0066] 5. Provide a list of ingredients:

[0067] Based on the menu, a list of ingredients such as "chicken, curry powder, various vegetables, and rice" will be displayed on the user's device.

[0068] 6. Use of delivery services:

[0069] The user selects a grocery delivery service, and the server initiates the process of delivering the groceries to the specified address.

[0070] This system allows users to plan economical and healthy meals without hassle and have ingredients delivered to their homes. This helps reduce food waste and loss, and also allows for better health management.

[0071] The following describes the processing flow.

[0072] Step 1: Gathering sale information

[0073] The server uses a script that runs periodically to retrieve special offer information from the APIs of partner retailers.

[0074] The server sends GET requests to each retailer's API endpoint and parses the JSON data received as a response.

[0075] The server analyzes the sale information and saves it to the database. This ensures that the latest sale information is always available.

[0076] Step 2: Enter user information

[0077] The user logs in to the application or website.

[0078] The device displays an input form to the user and collects allergy information, calorie restrictions, and other dietary conditions.

[0079] The user enters this information and clicks the submit button.

[0080] Step 3: Submit User Information

[0081] The device collects user information and sends it to the server as a POST request in JSON format.

[0082] For example, the device sends the following data to the server.

[0083] json

[0084] {

[0085] "user_id": "12345",

[0086] "allergies": ["nuts"],

[0087] "calorie_limit": 1500

[0088] }

[0089] Step 4: Saving User Information

[0090] The server stores the received user information in the database.

[0091] The server associates allergy information and calorie restrictions based on the user ID.

[0092] Step 5: Submit a menu generation request

[0093] The server sends a request to the generative AI based on the user's dietary requirements and the latest special offers.

[0094] Request data includes allergy information, calorie restrictions, and a list of sale items.

[0095] json

[0096] {

[0097] "allergies": ["nuts"],

[0098] "calorie_limit": 1500,

[0099] "Specials": ["Chicken", "Curry powder", "Vegetables"]

[0100] }

[0101] Step 6: Menu generation using generative AI

[0102] The generative AI analyzes the received request data and generates the optimal menu.

[0103] As an example, generate a menu consisting of "chicken curry, salad, and rice."

[0104] Step 7: Serving the menu

[0105] The server receives the generated menu data and sends it to the user's terminal.

[0106] The device displays the generated menu to the user. The user can view the menu details and cooking instructions.

[0107] Step 8: Generating the ingredient list

[0108] The server creates a list of necessary ingredients based on the generated menu.

[0109] The server sends the ingredient list to the user's terminal.

[0110] json

[0111] {

[0112] "items": ["Chicken", "Curry powder", "Vegetables", "Rice"]

[0113] }

[0114] Step 9: Choosing a delivery service

[0115] The user selects a grocery delivery service on their device.

[0116] The terminal notifies the server of the user's selection.

[0117] Step 10: Processing Orders and Deliveries

[0118] The server sends an order request, including a list of necessary ingredients and the delivery address, to the API of a partnered delivery service.

[0119] json

[0120] {

[0121] "address": "User's address",

[0122] "items": ["Chicken", "Curry powder", "Vegetables", "Rice"]

[0123] }

[0124] A delivery service receives the request and delivers the ingredients to the user's address.

[0125] This series of steps allows users to automatically obtain economical and healthy meal plans and necessary ingredients, which are then delivered to their homes. This eliminates the hassle of shopping and reduces food waste and unnecessary expenses.

[0126] (Example 1)

[0127] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0128] With existing systems, users have to manually collect sale information and create meal plans while considering allergy information and calorie restrictions, which is time-consuming. This is even more difficult for people who have difficulty going grocery shopping, such as the elderly and those with young children. This can lead to wasted food, food loss, and difficulties in managing one's health.

[0129] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0130] In this invention, the server includes means for collecting the latest sale information, means for obtaining allergy information and dietary conditions such as calorie restrictions entered by the user, means for generating a menu using a generation AI model based on the sale information and dietary conditions, means for creating a list of necessary ingredients based on the generated menu, and means for placing an order based on the ingredient list and having the ingredients delivered. This makes it possible for users to obtain economical and healthy menus without any hassle and have the ingredients delivered to their homes.

[0131] "Special sale information" refers to information about discounts and special offers on products that can be obtained from partner retailers.

[0132] "User information" refers to information about dietary conditions such as allergy information and calorie restrictions that users enter.

[0133] A "generative AI model" is an artificial intelligence model used to generate menus based on special offer information and user information.

[0134] A "menu" refers to a combination of meals suggested by an AI model based on special offer information and user data.

[0135] A "food ingredient list" is a list of ingredients needed based on the generated menu.

[0136] "Delivery service" refers to the means of arranging for the delivery of groceries ordered by a user to a specified address.

[0137] A "terminal" is an electronic device used by users to input information or to display generated menus and ingredient lists.

[0138] The "server" is a central computer system that performs tasks such as collecting special offer information, storing user information, operating the generated AI model, creating menus and ingredient lists, and arranging delivery methods.

[0139] A "database" is a system for storing and managing data such as special sale information and user information.

[0140] An "API" is an interface that allows different software components to interact with each other.

[0141] This invention is a system that automates everything from generating menus to purchasing and delivering ingredients, which are necessary when a user plans a meal. Specific embodiments of this system are shown below.

[0142] Gathering information on special offers

[0143] The server uses a script that automatically runs at a specific time (for example, 3 AM every day) to retrieve special offer information from the APIs of multiple partner retailers. This ensures that the latest special offer information is always stored in the database. The server uses the Python requests library to send GET requests to each retailer's API endpoint and stores the returned JSON data in a database such as MySQL®.

[0144] Entering user information

[0145] Users input their allergy information and dietary conditions such as calorie restrictions through the application or website. This information is received by the device and sent to the server as a POST request in JSON format. When the user enters the necessary information in a form on the browser and clicks the "Submit" button, the front-end JavaScript® code converts the data into JSON format and sends it to the server using the axios library.

[0146] User information storage

[0147] The server parses the user information it receives and stores it in a database. This makes it easy to reuse information that the user might need in the future. For example, the server parses JSON data received via a Flask or Django endpoint and saves it to a database such as MySQL.

[0148] Menu generation

[0149] The server generates a menu using a generative AI model (e.g., GPT-4®) based on special offer information and user information. The server sends a prompt to the OpenAI® API endpoint, and the AI ​​generates a menu based on that prompt. Examples of prompts include "Special offer information: chicken, curry powder, vegetables" and "User information: nut allergy, 1500 calorie limit." This generates a menu such as "chicken curry, salad, rice."

[0150] Menu

[0151] The generated menu is sent from the server to the user's device and displayed visually on the device. Users can view the menu and its details on devices such as PCs and smartphones. The server returns JSON data as a Flask or Django response, and the frontend displays the received data in HTML format.

[0152] Providing a list of ingredients

[0153] The server creates a list of necessary ingredients based on the generated menu and sends it to the user's terminal. An ingredient list such as "chicken, curry powder, various vegetables, rice" is displayed. The server selects the names of the necessary ingredients from the database to generate a list and sends it to the terminal in JSON format. The frontend receives the list data and displays it so that the user can visually confirm it.

[0154] Use of delivery service

[0155] When a user requests delivery, they make a selection through their device, and that information is sent to the server. The server sends the necessary information to the partner delivery service, and the groceries are delivered to the specified address. The server sends the "delivery address and list of required groceries" as a POST request in JSON format to the delivery company's API endpoint, and the actual delivery process is carried out.

[0156] This system allows users to plan economical and healthy meals without hassle and have ingredients delivered to their homes. This automation can reduce food waste and spoilage, and even improve health management.

[0157] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0158] Step 1: Gathering sale information

[0159] Processing flow:

[0160] The server automatically executes a script at a specific time (for example, 3 AM every day) to send GET requests to the APIs of multiple partner retailers. For example, the server accesses "https: / / api.storeA.com / sales" and "https: / / api.storeB.com / sales".

[0161] Specific actions:

[0162] The server sends a GET request using the Python requests library and receives JSON data returned as a response.

[0163] input:

[0164] The API endpoint URL for a specific retail store.

[0165] output:

[0166] Special sale information data in JSON format.

[0167] Data processing:

[0168] The server parses the received JSON data and executes an INSERT query to save it to the MySQL database.

[0169] Step 2: Enter user information

[0170] Processing flow:

[0171] Users enter dietary conditions such as allergy information and calorie restrictions through the application or website. For example, they might enter "I have a nut allergy" or "I need to limit myself to 1500 calories per day."

[0172] Specific actions:

[0173] The user enters information into a form in their browser and clicks the "Submit" button. JavaScript code on the frontend converts the input data into JSON format and sends a POST request to the server using the axios library.

[0174] input:

[0175] Allergy information and dietary conditions such as calorie restrictions entered by the user.

[0176] output:

[0177] User information data in JSON format.

[0178] Data processing:

[0179] The terminal parses the entered information in JSON format and sends it to the server.

[0180] Step 3: Saving User Information

[0181] Processing flow:

[0182] The server parses the received user information and stores it in the database. For example, it stores data corresponding to the fields "user_id", "allergies", and "calorie_limit".

[0183] Specific actions:

[0184] The server parses the JSON data received via the Flask or Django endpoint and saves it to the MySQL database using INSERT queries.

[0185] input:

[0186] User information data in JSON format.

[0187] output:

[0188] User information stored in the database.

[0189] Data processing:

[0190] The server parses the JSON data and inserts it into the corresponding database fields.

[0191] Step 4: Menu Generation

[0192] Processing flow:

[0193] The server sends prompts to a generating AI model (e.g., GPT-4) based on special offer information and user information, and the AI ​​generates a menu. For example, it might send prompts such as "Special offer information: Chicken, curry powder, vegetables" and "User information: Nut allergy, 1500 calorie limit".

[0194] Specific actions:

[0195] The server sends prompts to the OpenAI API endpoint using Python code and receives responses from the AI.

[0196] input:

[0197] A prompt message containing special offer information and user information.

[0198] output:

[0199] The generated menu data.

[0200] Data processing:

[0201] The server generates a prompt and sends it to the AI ​​model. The AI's response is analyzed and saved in JSON format.

[0202] Step 5: Serving the menu

[0203] Processing flow:

[0204] The generated menu is sent from the server to the user's device and displayed visually on the device. Users can check the details of the menu on devices such as PCs and smartphones.

[0205] Specific actions:

[0206] The server returns JSON data as a response from Flask or Django, which is then parsed on the frontend and displayed in HTML format.

[0207] input:

[0208] The generated menu data.

[0209] output:

[0210] Menu information displayed on the user's terminal.

[0211] Data processing:

[0212] The device parses the JSON data and generates HTML for visual display.

[0213] Step 6: Provide the ingredient list

[0214] Processing flow:

[0215] The server creates a list of necessary ingredients based on the generated menu and sends it to the user's terminal. For example, it might generate a list such as "chicken, curry powder, various vegetables, rice."

[0216] Specific actions:

[0217] The server selects the necessary ingredient information from the database, generates a list, and sends it to the terminal in JSON format.

[0218] input:

[0219] The generated menu data.

[0220] output:

[0221] A list of ingredients displayed on the user's terminal.

[0222] Data processing:

[0223] The server retrieves ingredient information from the database, creates a list, and sends it.

[0224] Step 7: Use a delivery service

[0225] Processing flow:

[0226] When a user requests grocery delivery, they send their selection information to a server via their device. The server then transmits the necessary information to a partner delivery service, and the groceries are delivered to the specified address.

[0227] Specific actions:

[0228] The user clicks the "Request Delivery" button in the application's UI, and the frontend sends a POST request to the server. The server then sends the delivery information to the delivery company's API endpoint via a POST request.

[0229] input:

[0230] User delivery preference information.

[0231] output:

[0232] Delivery information sent to the courier company.

[0233] Data processing:

[0234] The server analyzes the user's delivery information and sends it to the delivery company's API in the appropriate format.

[0235] (Application Example 1)

[0236] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0237] Traditional systems required users to manually search for sale information and create meal plans considering allergy information and calorie restrictions. Furthermore, it was difficult to effectively utilize sale information to generate economical meal plans, and the procedures for using food delivery services were cumbersome. Therefore, there was a need to simultaneously achieve effective use of sale information and health management.

[0238] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0239] In this invention, the server includes means for acquiring the latest special offer information, means for acquiring allergy information and dietary conditions such as calorie restrictions entered by the user, means for utilizing a generative AI model that generates menus based on the special offer information and the dietary conditions, means for creating a list of necessary ingredients based on the generated menus, and means for placing orders and having the ingredients delivered based on the ingredient list. As a result, users can effectively utilize special offer information, automatically generate menus suited to their individual dietary conditions, and easily purchase and have the necessary ingredients delivered.

[0240] "Means of obtaining the latest sale information" refers to a system or method that can regularly collect and store sale information provided by supermarkets and other retailers.

[0241] "Means for obtaining user-entered allergy information and dietary conditions such as calorie restrictions" refers to an interface for users to input individual dietary restrictions, such as allergies to specific foods or calorie restrictions, and a system for processing that information.

[0242] "Means of using a generative AI model that generates menus based on the aforementioned special offer information and the aforementioned meal conditions" refers to a system or method that uses an artificial intelligence model that automatically generates appropriate menus based on collected special offer information and the user's meal conditions as input.

[0243] "Means for creating a list of necessary ingredients based on the generated menu" refers to a system or method that has the function of compiling a list of necessary ingredients based on the generated menu.

[0244] "Means of ordering and delivering ingredients based on the aforementioned ingredient list" refers to a system or method that allows a user to order ingredients online based on a generated ingredient list and have those ingredients delivered to a specified address.

[0245] This invention is a system that allows users to plan healthy and economical meals without hassle and efficiently obtain the necessary ingredients. The system consists of a server, a user terminal, and partner retailers and delivery services.

[0246] First, the server periodically retrieves special offer information from multiple partner retailers. This information is collected via APIs provided by the retailers and stored in the server's database. The special offer information is retrieved in JSON format, and the database always maintains the most up-to-date information.

[0247] Next, the user enters allergy information and dietary conditions such as calorie restrictions into a dedicated application via their smartphone or computer. This information is sent from the device to a server and stored in a database.

[0248] The server uses a generative AI model to generate a menu tailored to the user, based on saved sale information and the user's dietary conditions. In this case, the generative AI model uses the sale information and user information as input. For example, based on user information such as "nut allergy, 1500 calories per day" and sale items such as "chicken, curry powder, vegetables," the model suggests a menu such as "chicken curry, salad, rice."

[0249] Furthermore, the server creates a list of necessary ingredients based on the generated menu and sends this list to the user's device. This list is displayed in a format that can be visually confirmed on the user's device.

[0250] When a user wants groceries delivered, they place an order through their device. This information is sent to the server, which uses the API of a partnered delivery service to arrange for the groceries to be delivered to the specified address.

[0251] For example, a menu is generated based on the user's input conditions, such as "nut allergy, 1500 calories per day," and special offer information, and a list of corresponding ingredients is displayed on the terminal. If the user selects ingredient delivery, the server automatically places an order with the delivery service, and the ingredients are delivered to the user's address.

[0252] An example of a prompt statement is as follows:

[0253] "User information: Has a nut allergy, daily calorie intake of 1500 kcal. Special offer information: Chicken, curry powder, vegetables."

[0254] This system allows users to efficiently plan healthy meals using sale information and easily obtain the necessary ingredients. As a result, it reduces the effort required from users, facilitates the effective use of sale information, improves health management, and reduces food waste.

[0255] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0256] Step 1:

[0257] The server periodically retrieves special offer information from the APIs of partner retailers. Specifically, the server sends a GET request to a designated URL of a retailer at a fixed time each day and receives the special offer information in JSON format. This JSON data is stored in a database. The input data is the special offer information retrieved from the retailer, and the output data is the special offer information stored in the database.

[0258] Step 2:

[0259] Users input their allergy information and dietary conditions, such as calorie restrictions, through a smartphone or computer application. This information is sent from the device to the server and stored in a database in JSON format. The input data is the user's dietary conditions, and the output data is the user's dietary conditions stored in the database.

[0260] Step 3:

[0261] The server sends special offer information retrieved from the database and the user's dietary conditions as input to the generating AI model. At this time, it generates a prompt message and makes a request to the AI ​​model. For example, it sends data in the format: "User information: Nut allergy, 1500 calories per day. Special offer information: Chicken, curry powder, vegetables." The generating AI model proposes an appropriate menu, which the server receives. The input data consists of the user's dietary conditions and special offer information, while the output data is the generated menu.

[0262] Step 4:

[0263] The server creates a list of necessary ingredients based on the generated menu. This list is generated in JSON format and sent to the user's terminal. Specifically, it analyzes the menu data obtained from the generation AI model and lists the necessary ingredients. The input data is the generated menu, and the output data is the ingredient list.

[0264] Step 5:

[0265] The user reviews the displayed list of ingredients and selects their delivery preferences through the application. This information is sent from the terminal to the server. The input data is the user's delivery preference information, and the output data is the delivery preference information stored on the server.

[0266] Step 6:

[0267] The server places an order with a partner delivery service via its API, based on the user's delivery preferences and ingredient list. Specifically, it converts the delivery address and required ingredient list into an API request format and sends it as a POST request. The delivery service then returns confirmation that the order has been received. The input data consists of the delivery preferences and ingredient list, while the output data is the order confirmation sent to the delivery service.

[0268] Through these steps, users can easily take advantage of special offers, create healthy and economical meal plans, and efficiently obtain the necessary ingredients.

[0269] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0270] This invention is a system that targets people who cook at home and generates economical and healthy menus that take into account supermarket sale information and the user's dietary conditions (allergy information and calorie restrictions). Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to suggest menus that are tailored to the user's mental state. In addition, it features integration with a delivery service, making it possible to deliver ingredients to people who have difficulty going shopping, such as those raising children or the elderly.

[0271] Program processing flow

[0272] Emotion recognition by an emotion engine

[0273] This system includes an emotion engine that recognizes the user's emotions. The emotion engine analyzes facial recognition data and voice data acquired from the user's device to determine the user's emotional state in real time.

[0274] Gathering information on special offers

[0275] The server retrieves special offer information from APIs of multiple partner retailers. This information is updated daily and stored in a database. The server sends requests to each retailer's API endpoint, parses the JSON data received as a response, and stores it in the database.

[0276] Entering and saving user information

[0277] The user inputs allergy information, calorie restrictions, current emotional state, etc. through a website or application. The terminal collects this information and sends it to the server. The server saves the received information in a database.

[0278] Recipe generation

[0279] Based on the special offer information obtained by the server, the user's dietary conditions, and the user's emotional state recognized by the emotion engine, a recipe is generated using generative AI. For example, when the user is feeling stressed, a recipe using ingredients with a relaxing effect is proposed.

[0280] Recipe provision and adjustment

[0281] The generated recipe is sent from the server to the user's terminal and displayed on the terminal. The user can check the recipe that accommodates allergies and calorie restrictions. Also, a menu based on the emotional state is proposed.

[0282] Ingredient list generation

[0283] The server creates a list of required ingredients based on the generated recipe and sends it to the user's terminal. The user can check the list and proceed with the order.

[0284] Use of delivery service

[0285] When the user selects a delivery service, the terminal notifies the server of that information. The server sends the required ingredient list and the delivery address to the API of the partnered delivery service and proceeds with the procedure to deliver the ingredients to the user's address.

[0286] Specific example

[0287] 1. Emotion recognition and acquisition of special offer information:

[0288] The user uses the device's camera and microphone to recognize their emotions. The emotion engine analyzes the user's facial recognition data and voice data and recognizes it as "stress."

[0289] The server retrieves the latest sale information from "Store A" and "Store B," and items such as "chicken," "curry powder," and "vegetables" are listed as sale items.

[0290] 2. Entering and saving user information:

[0291] The user enters "I have a nut allergy" and "I'm limiting myself to 1500 calories a day" into the application.

[0292] The server saves this information to the database.

[0293] 3. Menu generation:

[0294] Based on sale information, the user's meal preferences, and the results of the emotion engine, the server uses generative AI to generate a menu of "relaxing chicken soup, salad, and rice."

[0295] 4. Menu provision and adjustment:

[0296] The server sends the generated menu to the user's terminal, and the user checks the menu ("chicken soup, salad, rice") and the explanation of its relaxing effects in the application.

[0297] 5. Generating the ingredient list:

[0298] Based on the menu, a list of ingredients such as "chicken, curry powder, various vegetables, and rice" is displayed on the user's device.

[0299] 6. Use of delivery services:

[0300] The user selects a grocery delivery service, and the server sends the necessary information to the partnered delivery service.

[0301] A delivery service delivers groceries to the user's address.

[0302] As described above, this system comprehensively considers the user's current emotional state, health condition, and dietary conditions, and provides specific and practical meal plans and ingredients. As a result, the user can lead a healthy diet both physically and mentally.

[0303] The processing flow will be described below.

[0304] Step 1: Collection of special offer information

[0305] Using a script that the server executes regularly, obtain special offer information from the APIs of multiple partner stores. Specifically, the server sends a GET request to each API endpoint at 3:00 am every day, and analyzes the JSON data received as a response.

[0306] The server saves the received special offer information in the database. As a result, the database always holds the latest special offer information.

[0307] Step 2: Input of user information

[0308] The user logs in through an application or website.

[0309] The terminal displays an input form to the user and collects allergy information, calorie limit, and other dietary conditions. For example, input information such as "has nut allergy" and "1500 calorie limit per day".

[0310] The user checks the collected information and clicks the send button.

[0311] Step 3: Transmission of user information

[0312] The terminal sends the user information collected in JSON format to the server as a POST request.

[0313] For example, the device sends the following data to the server.

[0314] json

[0315] {

[0316] "user_id": "12345",

[0317] "allergies": ["nuts"],

[0318] "calorie_limit": 1500

[0319] }

[0320] Step 4: Saving User Information

[0321] The server stores the received user information in a database. Allergy information and calorie restrictions are associated based on the user ID.

[0322] Step 5: Emotion recognition by the emotion engine

[0323] The device activates its function to acquire the user's facial recognition data and voice data.

[0324] The user shows their face to the device's camera and microphone and speaks.

[0325] The device sends facial recognition data and voice data to the emotion engine, which then analyzes it.

[0326] The server receives the analysis results and determines the user's current emotional state. For example, it might determine that the user is "stressed."

[0327] Step 6: Submit a menu generation request

[0328] The server sends a request to the generative AI based on the user's meal conditions, the results of the emotion engine, and special offer information.

[0329] Request data includes allergy information, calorie restrictions, a list of sale items, and the user's current emotional state.

[0330] json

[0331] {

[0332] "allergies": ["nuts"],

[0333] "calorie_limit": 1500,

[0334] "specials": ["chicken", "curry powder", "vegetables"],

[0335] "emotion": "stress"

[0336] }

[0337] Step 7: Menu generation using generative AI

[0338] The generative AI analyzes the received request data and generates a menu that is appropriate for the user's emotional state.

[0339] For example, if the emotion engine's result is "stress," it will generate a menu of "chicken soup, salad, and rice" which have a relaxing effect.

[0340] Step 8: Serving the menu

[0341] The server receives the generated menu data and sends it to the user's terminal.

[0342] The device displays the menu to the user, who can then view the details. For example, a menu such as "chicken soup, salad, and rice" and a description of its relaxing effects might be displayed.

[0343] Step 9: Generating the ingredient list

[0344] The server creates a list of necessary ingredients based on the generated menu.

[0345] The server sends a list of ingredients to the user's device. For example, an ingredient list such as "chicken, curry powder, various vegetables, and rice" will be displayed.

[0346] Step 10: Choosing a delivery service

[0347] The user selects a grocery delivery service.

[0348] The terminal notifies the server of its selection.

[0349] Step 11: Processing Orders and Deliveries

[0350] The server sends an order request, including a list of necessary ingredients and the delivery address, to the API of a partnered delivery service.

[0351] json

[0352] {

[0353] "address": "User's address",

[0354] "items": ["Chicken", "Curry powder", "Vegetables", "Rice"]

[0355] }

[0356] A delivery service receives the request and delivers the ingredients to the user's address.

[0357] This series of steps allows users to obtain economical and healthy meal plans and necessary ingredients without requiring specialized knowledge or time. Furthermore, meal suggestions that take into account the user's emotional state can comprehensively support their physical and mental health.

[0358] (Example 2)

[0359] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal".

[0360] Conventional menu generation systems only considered the user's dietary conditions and sale information, but did not take into account the user's emotional state. Therefore, they could not suggest menus that suited the user's mental state and could not fully meet their needs. Furthermore, few systems integrated grocery delivery services for users who could not go shopping. As a result, users were not receiving significant benefits.

[0361] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0362] In this invention, the server includes means for acquiring the latest sale information, means for acquiring allergy information and dietary conditions such as calorie restrictions entered by the user, means for recognizing the user's emotional state, means for using a generative AI model that generates a menu based on the sale information, the dietary conditions, and the emotional state, means for creating a list of necessary ingredients based on the generated menu, and means for placing an order based on the ingredient list and having the ingredients delivered. This enables the integration of menu suggestions tailored to the user's mental state and an ingredient delivery service.

[0363] 1. "Latest special offer information" refers to the latest sales information, such as prices and special offers, obtained from our partner retailers.

[0364] 2. "Allergy information" refers to information about a user's allergic reactions to specific foods.

[0365] 3. "Calorie restriction" refers to information about the limit on the total amount of calories a user can consume in a day.

[0366] 4. "Emotional state" refers to the psychological state determined by analyzing the user's facial recognition data and voice data.

[0367] 5. A "generative AI model" is an artificial intelligence model that generates a desired output when specific conditions are input.

[0368] 6. A "menu" is a specific meal plan generated based on the user's eating conditions and emotional state.

[0369] 7. A "food ingredient list" is a list containing the types and quantities of ingredients needed for the generated menu.

[0370] 8. A "delivery service" is a service in which a user orders selected ingredients and has them delivered to a specified address.

[0371] 9. A "terminal" is an electronic device (e.g., a smartphone, tablet, or personal computer) used by a user to input information or check results.

[0372] 10. A "server" is a central computer that manages and processes data for the entire system and provides various services.

[0373] This invention is a system that generates economical and healthy menus for users who cook at home, taking into account supermarket sale information and the user's dietary conditions (allergy information and calorie restrictions). Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to suggest menus that are tailored to the user's mental state. In addition, it features integration with a delivery service, enabling the delivery of ingredients to people who have difficulty going shopping, such as those raising children or the elderly.

[0374] This system includes the following main components: servers, terminals, and the hardware and software used by users.

[0375] Hardware and software to be used

[0376] 1. Server:

[0377] The server retrieves special offer information from APIs of multiple retailers and stores it in a database. It also generates meal plans using a generative AI model based on the user's dietary conditions and emotional state.

[0378] Software used: API clients, database management systems, generative AI models, etc.

[0379] 2. Terminal:

[0380] The terminal sends user data input to a server and displays the generated menu and ingredient list. It also uses a camera and microphone for emotion recognition.

[0381] Hardware to use: Smartphones, tablets, and personal computers.

[0382] Software used: emotion recognition engine, web browser or application, etc.

[0383] 3. User:

[0384] Users input allergy information and calorie restrictions via their device, and their emotional state is recognized. They also review suggested menus and ingredient lists, and select a delivery service as needed.

[0385] Specific examples of program processing

[0386] 1. Emotion recognition and acquisition of special offer information:

[0387] The user uses their device's camera and microphone to perform emotion recognition. The emotion engine determines the user's emotions in real time and recognizes them as, for example, "stress."

[0388] The server retrieves the latest sale information from partner retailers, and items such as "chicken," "curry powder," and "vegetables" are listed as sale items.

[0389] 2. Entering and saving user information:

[0390] Users enter information such as "I have a nut allergy" or "I have a 1500 calorie limit per day" through the application.

[0391] The device collects this information and sends it to the server.

[0392] The server saves the received information to the database.

[0393] 3. Menu generation:

[0394] The server generates prompt messages for the generative AI based on sale information, the user's dietary conditions, and the results of the emotion engine. Example of a prompt message: "Suggest a menu that the user is stressed. Sale information includes chicken, vegetables, and bread, and the user has a nut allergy."

[0395] The server inputs prompt text into a generative AI model, and a menu is generated, for example, "relaxing chicken soup, salad, and rice."

[0396] 4. Menu provision and adjustment:

[0397] The server sends the generated menu to the user's terminal.

[0398] The user reviews the suggested menu on their device and makes adjustments as needed.

[0399] 5. Generating the ingredient list:

[0400] The server creates a list of necessary ingredients based on the generated menu.

[0401] The server sends the ingredient list to the user's terminal.

[0402] 6. Use of delivery services:

[0403] The user selects a grocery delivery service through the app.

[0404] The device notifies the server of that information.

[0405] The server sends a list of necessary ingredients and the delivery address to the API of the partnered delivery service, and then processes the delivery.

[0406] Through the processing steps described above, this system comprehensively considers the user's current emotional state, health condition, and dietary requirements, and provides specific and practical menus and ingredients. As a result, the user can lead a healthy lifestyle both physically and mentally.

[0407] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0408] Step 1: Emotion Recognition

[0409] The user uses the device's camera and microphone to allow the system to recognize their emotional state. The device acquires facial recognition data and voice data in real time. This input data includes the user's facial expressions and voice tone.

[0410] The device sends this data to the emotion engine, which then performs analysis. For example, by analyzing the user's facial muscle movements and voice pitch, the emotion engine determines the user's emotional state as "stressed," "happy," or "tired."

[0411] Step 2: Gathering information on special offers

[0412] The server sends requests to the APIs of partner retailers to retrieve the latest sale information. This input data consists of the API endpoints of each retailer.

[0413] The server receives special offer information (e.g., "Chicken: 200 yen / 100g", "Curry powder: 100 yen / bag", "Vegetables: 150 yen / bag") in JSON format, analyzes it, and then stores the analysis results in the database.

[0414] Step 3: Enter and save user information

[0415] Users input their allergy information and dietary restrictions, such as calorie limits, through a website or application. Specific input information includes "I have a nut allergy" and "I have a 1500 calorie limit per day."

[0416] The device collects this information and sends it to the server.

[0417] The server saves the received user information to the database. This data processing stores the user's individual conditions in the database.

[0418] Step 4: Menu Generation

[0419] The server retrieves special offer information, user dietary conditions, and emotion engine results from the database. Based on this information, the generative AI generates prompt messages. The input data includes special offer information, allergy information, calorie restrictions, and emotional state.

[0420] Example of a prompt message: "When the user is feeling stressed, suggest a recommended meal. Special offers include chicken, vegetables, and bread, and the user has a nut allergy."

[0421] The server inputs a prompt message into the generative AI model and receives the generated menu. For example, a menu such as "relaxing chicken soup, salad, and rice" might be output.

[0422] Step 5: Menu provision and adjustment

[0423] The server sends the generated menu to the user's terminal. The output data is the generated menu.

[0424] The user reviews the suggested menu on their device and makes adjustments as needed. For example, the user reviews a menu of "chicken soup, salad, and rice" and its explanation of its relaxing effects.

[0425] Step 6: Generating the ingredient list

[0426] The server creates a list of necessary ingredients based on the generated menu. The input data is the generated menu.

[0427] The server sends a list of ingredients it has created to the user's terminal. The ingredient list includes "chicken, curry powder, various vegetables, and rice."

[0428] Step 7: Use a delivery service

[0429] Users select grocery delivery through the application.

[0430] The device notifies the server of that information.

[0431] The server sends the necessary information (food list and delivery address) to the API of the partnered delivery service, and the delivery process is initiated. Specifically, the server sends the food list to the API, and the package is delivered to the user's address.

[0432] In this way, the system comprehensively considers the user's emotional state, dietary conditions, and special offer information to provide the optimal menu, and further supports the easy acquisition of ingredients through home delivery services.

[0433] (Application Example 2)

[0434] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0435] Existing home cooking support systems offered menu suggestions that took into account users' allergy information and dietary restrictions such as calorie limits, but they lacked the functionality to consider the user's emotional state and suggest menus that responded to it in real time. Furthermore, there is a need to make shopping at physical stores more comfortable and effective by efficiently utilizing sale items and suggesting economical and healthy menus.

[0436] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring the latest sale information, means for acquiring allergy information and dietary conditions such as calorie restrictions entered by the user, emotion recognition means for recognizing the user's emotions, means for generating a menu based on the sale information, the dietary conditions, and the emotion recognition results, means for creating a list of necessary ingredients based on the generated menu, means for placing an order based on the ingredient list and having the ingredients delivered, and means for displaying the generated menu on the user's terminal. This enables menu suggestions that take into account the user's current emotional state and efficient shopping at physical stores.

[0437] "Latest special offer information" refers to special price information for currently available products, obtained from multiple partner retailers.

[0438] "User-entered allergy information" refers to information about a user's allergies that they provide to the system.

[0439] "Calorie restriction" refers to the upper limit of daily calorie intake set by the user.

[0440] "Dietary conditions" refer to specific conditions or restrictions related to diet, such as the user's allergy information or calorie restrictions.

[0441] "Emotion recognition technology" refers to technology that analyzes and recognizes a user's emotions in real time.

[0442] The "means for generating menus" refer to a function that suggests appropriate meal menus based on acquired special offer information, meal conditions, and emotion recognition results.

[0443] The "list of necessary ingredients" is a list of all the ingredients needed to actually prepare the generated menu.

[0444] "A means of placing an order and having ingredients delivered" refers to a function that allows users to purchase ingredients they specify and arrange for them to be delivered to a designated address.

[0445] "User's device" refers to a smartphone, smart glasses, or any other electronic device used by the user.

[0446] A "generative AI model" is an artificial intelligence model that automatically generates menus tailored to user needs based on diverse data.

[0447] To implement this invention, a server, user terminals (smartphones and smart glasses), and API connections from multiple retailers are utilized. The specific hardware and software, data processing, and data calculations required for this are described below.

[0448] Hardware and software

[0449] Server: A computer server installed in a cloud environment.

[0450] User devices: Smartphones, smart glasses (e.g., Google® Glass®)

[0451] Emotion recognition engine: Microsoft® Azure® Emotion API

[0452] Database management system: MySQL

[0453] Special Sale Information Acquisition API: APIs for various supermarkets

[0454] Generative AI: OpenAI GPT-3 (registered trademark)

[0455] Data processing and data calculation

[0456] Emotion recognition and acquisition of sale information

[0457] The device's camera and microphone are used to send user facial recognition data and voice data to an emotion recognition engine. The emotion recognition engine analyzes the user's emotions and displays the results on the device in real time. The server retrieves the latest special offer information from the APIs of partner retailers, receives it in JSON format, and stores it in a database.

[0458] Entering and saving user information

[0459] Upon initial login, users enter allergy information and calorie restrictions via their device. This information is sent to the server and stored in the database.

[0460] Menu generation

[0461] The server retrieves special offer information, user health information, and emotion recognition results from the database. Then, it inputs prompts like the following into the generative AI to generate a menu.

[0462] Example: Prompt text to input to a generative AI

[0463] The user has a dairy allergy and a daily calorie restriction of 1800 calories. The user's current emotional state is fatigue. Please suggest a meal consisting of stir-fried chicken and broccoli, a salad, and whole-wheat bread, all of which are known to help with fatigue recovery.

[0464] The generated menu is saved to a database by the server.

[0465] Menu provision and adjustment

[0466] Users can view the generated menu by wearing smart glasses or using a smartphone app in the store. They can adjust the menu as needed.

[0467] Generating an ingredient list

[0468] Based on the proposed menu, the server extracts the necessary ingredients from the database and displays them on the user's terminal.

[0469] Use of delivery service

[0470] When a user selects a delivery service, the server retrieves the necessary information from the database and sends it to the API of the partnered delivery service. The delivery service then delivers the specified ingredients to the user's address.

[0471] As a concrete example, if a user wears smart glasses and the emotion recognition engine detects "fatigue," the server retrieves "chicken," "broccoli," and "olive oil" as sale items from "Supermarket A." Based on the information the user has entered, such as "dairy allergy" and "1800 calorie limit per day," the generative AI generates a menu of "stir-fried chicken and broccoli, salad, and whole wheat bread." The user confirms this menu, and "chicken, broccoli, olive oil, and whole wheat bread" are displayed as necessary ingredients. If the user selects a delivery service, the ingredients are delivered to their home.

[0472] As a result, users can receive menu suggestions that take their emotional state into consideration and achieve efficient shopping at physical stores.

[0473] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0474] Step 1:

[0475] Users log in to the system via smart glasses or smartphones. After logging in, they enter initial settings such as allergy information and dietary conditions including calorie restrictions. The entered information is sent from the device to the server and stored in the database.

[0476] Input: User allergy information, calorie restriction settings

[0477] Data processing: Information collection and format conversion

[0478] Output: User information stored in the database

[0479] Step 2:

[0480] The user uses the smart glasses' camera and microphone to send facial recognition data and voice data to the emotion recognition engine. The emotion recognition engine analyzes this data and recognizes the user's emotional state in real time. The recognition results are displayed on the device.

[0481] Input: User's facial recognition data, voice data

[0482] Data processing: Data analysis using an emotion recognition engine.

[0483] Output: Real-time emotional state

[0484] Step 3:

[0485] The server retrieves the latest sale information from the APIs of each partner retailer. The retrieved sale information is received in JSON format, the server analyzes the data, and saves the necessary information to the database.

[0486] Input: Special offer information from retailer API

[0487] Data processing: Parsing JSON data and storing it in a database.

[0488] Output: Special sale information stored in the database

[0489] Step 4:

[0490] The server retrieves the user's dietary conditions (allergy information, calorie restrictions), emotion recognition results, and special offer information from the database. Based on this information, it generates prompt messages, which are then input into a generative AI to generate a menu.

[0491] Input: User's dietary conditions, emotional state, special offer information

[0492] Data processing: Prompt processing by generative AI

[0493] Output: Generated menu

[0494] Examples of prompt statements:

[0495] The user has a dairy allergy and a daily calorie restriction of 1800 calories. The user's current emotional state is fatigue. Please suggest a meal consisting of stir-fried chicken and broccoli, a salad, and whole-wheat bread, all of which are known to help with fatigue recovery.

[0496] Step 5:

[0497] The server sends the generated menu to the user's terminal. The user can review the menu and make adjustments as needed. The adjustment results are then sent back to the server.

[0498] Input: Generated menu

[0499] Data processing: Information transmission and user adjustments.

[0500] Output: Adjusted final menu

[0501] Step 6:

[0502] The server creates a list of necessary ingredients based on the final menu and sends it to the terminal. The user reviews the list and selects the ingredients they want to order.

[0503] Input: Final Menu

[0504] Data processing: Extraction and creation of a list of necessary ingredients.

[0505] Output: Ingredient list

[0506] Step 7:

[0507] When a user selects grocery delivery, the device notifies the server of this information. The server then sends the necessary information to the API of its partner delivery service and initiates the process of delivering the groceries to the user's address.

[0508] Input: Ingredient list, delivery selection information

[0509] Data processing: Sending data to delivery service APIs

[0510] Output: Food items delivered to the user's address

[0511] The above outlines the specific processing steps of the program of this invention. Through these steps, users can achieve healthier and more efficient shopping based on personalized suggestions that take their emotional state into consideration.

[0512] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0513] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0514] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0515] [Second Embodiment]

[0516] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0517] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0518] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0519] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0520] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0521] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0522] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0523] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0524] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0525] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0526] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0527] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0528] This invention is a system that targets people who cook at home and generates economical and healthy menus by taking into account supermarket sale information and dietary conditions specified by the user (allergy information and calorie restrictions). Furthermore, by linking with delivery services, it is possible to deliver ingredients to people who have difficulty going shopping, such as those raising children or the elderly.

[0529] This system mainly consists of the following elements:

[0530] 1. Gathering information on special offers

[0531] 2. Entering and saving user information

[0532] 3. Menu generation

[0533] 4. Providing a list of ingredients

[0534] 5. Use of delivery services

[0535] Program processing flow

[0536] Gathering information on special offers

[0537] The server uses a script that runs at specific times to retrieve special offer information from the APIs of multiple partner retailers. For example, the server sends a GET request to a specific URL at 3 AM every day and saves the received JSON data to the database. In this way, the latest special offer information is always stored in the database.

[0538] Entering user information

[0539] Users input dietary information such as allergy details and calorie restrictions through applications or websites. The device receives this information and sends it to the server. For example, a user might enter information such as "I have a nut allergy" and "I have a 1500 calorie limit per day" into an input form. The device then sends this data to the server as a POST request in JSON format.

[0540] User information storage

[0541] The server stores the received user information in a database. This makes it easy to reuse information that the user may need in the future.

[0542] Menu generation

[0543] The server uses a generative AI to generate menus based on user information and acquired sale information. For example, the server sends the aforementioned user information and sale information to the AI, which then suggests a menu like "chicken curry, salad, and rice."

[0544] Menu

[0545] The generated menu is sent from the server to the user's device and displayed visually on the device. Users can check the menu and its details on devices such as PCs and smartphones.

[0546] Providing a list of ingredients

[0547] Based on the menu, the server creates a list of necessary ingredients and sends it to the user's terminal. For example, an ingredient list such as "chicken, curry powder, various vegetables, rice" will be displayed.

[0548] Use of delivery service

[0549] When a user requests delivery, they make a selection through their device, and that information is sent to the server. The server then sends the necessary information to the partner delivery service, and the groceries are delivered to the specified address. For example, the server might send the "delivery address and list of required groceries" as a POST request to the delivery company's API.

[0550] Specific example

[0551] In reality, the following scenario is possible:

[0552] 1. Gathering information on special offers:

[0553] The server retrieves special offer information from "Store A" and "Store B," and items such as "chicken," "curry powder," and "vegetables" are listed as special offer products.

[0554] 2. Entering and saving user information:

[0555] The user accesses the website and enters "nut allergy" as allergy information and "1500 calories" as calorie restriction.

[0556] 3. Menu generation:

[0557] Based on the aforementioned special offer information and user information, the server uses a generative AI to generate a menu consisting of "chicken curry, salad, and rice."

[0558] 4. Menu provision:

[0559] The server sends this menu to the user's device, and the user checks the menu on their smartphone.

[0560] 5. Provide a list of ingredients:

[0561] Based on the menu, a list of ingredients such as "chicken, curry powder, various vegetables, and rice" will be displayed on the user's device.

[0562] 6. Use of delivery services:

[0563] The user selects a grocery delivery service, and the server initiates the process of delivering the groceries to the specified address.

[0564] This system allows users to plan economical and healthy meals without hassle and have ingredients delivered to their homes. This helps reduce food waste and loss, and also allows for better health management.

[0565] The following describes the processing flow.

[0566] Step 1: Gathering sale information

[0567] The server uses a script that runs periodically to retrieve special offer information from the APIs of partner retailers.

[0568] The server sends GET requests to each retailer's API endpoint and parses the JSON data received as a response.

[0569] The server analyzes the sale information and saves it to the database. This ensures that the latest sale information is always available.

[0570] Step 2: Enter user information

[0571] The user logs in to the application or website.

[0572] The device displays an input form to the user and collects allergy information, calorie restrictions, and other dietary conditions.

[0573] The user enters this information and clicks the submit button.

[0574] Step 3: Submit User Information

[0575] The device collects user information and sends it to the server as a POST request in JSON format.

[0576] For example, the device sends the following data to the server.

[0577] json

[0578] {

[0579] "user_id": "12345",

[0580] "allergies": ["nuts"],

[0581] "calorie_limit": 1500

[0582] }

[0583] Step 4: Saving User Information

[0584] The server stores the received user information in the database.

[0585] The server associates allergy information and calorie restrictions based on the user ID.

[0586] Step 5: Submit a menu generation request

[0587] The server sends a request to the generative AI based on the user's dietary requirements and the latest special offers.

[0588] Request data includes allergy information, calorie restrictions, and a list of sale items.

[0589] json

[0590] {

[0591] "allergies": ["nuts"],

[0592] "calorie_limit": 1500,

[0593] "Specials": ["Chicken", "Curry powder", "Vegetables"]

[0594] }

[0595] Step 6: Menu generation using generative AI

[0596] The generative AI analyzes the received request data and generates the optimal menu.

[0597] As an example, generate a menu consisting of "chicken curry, salad, and rice."

[0598] Step 7: Serving the menu

[0599] The server receives the generated menu data and sends it to the user's terminal.

[0600] The device displays the generated menu to the user. The user can view the menu details and cooking instructions.

[0601] Step 8: Generating the ingredient list

[0602] The server creates a list of necessary ingredients based on the generated menu.

[0603] The server sends the ingredient list to the user's terminal.

[0604] json

[0605] {

[0606] "items": ["Chicken", "Curry powder", "Vegetables", "Rice"]

[0607] }

[0608] Step 9: Choosing a delivery service

[0609] The user selects a grocery delivery service on their device.

[0610] The terminal notifies the server of the user's selection.

[0611] Step 10: Processing Orders and Deliveries

[0612] The server sends an order request, including a list of necessary ingredients and the delivery address, to the API of a partnered delivery service.

[0613] json

[0614] {

[0615] "address": "User's address",

[0616] "items": ["Chicken", "Curry powder", "Vegetables", "Rice"]

[0617] }

[0618] A delivery service receives the request and delivers the ingredients to the user's address.

[0619] This series of steps allows users to automatically obtain economical and healthy meal plans and necessary ingredients, which are then delivered to their homes. This eliminates the hassle of shopping and reduces food waste and unnecessary expenses.

[0620] (Example 1)

[0621] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0622] With existing systems, users have to manually collect sale information and create meal plans while considering allergy information and calorie restrictions, which is time-consuming. This is even more difficult for people who have difficulty going grocery shopping, such as the elderly and those with young children. This can lead to wasted food, food loss, and difficulties in managing one's health.

[0623] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0624] In this invention, the server includes means for collecting the latest sale information, means for obtaining allergy information and dietary conditions such as calorie restrictions entered by the user, means for generating a menu using a generation AI model based on the sale information and dietary conditions, means for creating a list of necessary ingredients based on the generated menu, and means for placing an order based on the ingredient list and having the ingredients delivered. This makes it possible for users to obtain economical and healthy menus without any hassle and have the ingredients delivered to their homes.

[0625] "Special sale information" refers to information about discounts and special offers on products that can be obtained from partner retailers.

[0626] "User information" refers to information about dietary conditions such as allergy information and calorie restrictions that users enter.

[0627] A "generative AI model" is an artificial intelligence model used to generate menus based on special offer information and user information.

[0628] A "menu" refers to a combination of meals suggested by an AI model based on special offer information and user data.

[0629] A "food ingredient list" is a list of ingredients needed based on the generated menu.

[0630] "Delivery service" refers to the means of arranging for the delivery of groceries ordered by a user to a specified address.

[0631] A "terminal" is an electronic device used by users to input information or to display generated menus and ingredient lists.

[0632] The "server" is a central computer system that performs tasks such as collecting special offer information, storing user information, operating the generated AI model, creating menus and ingredient lists, and arranging delivery methods.

[0633] A "database" is a system for storing and managing data such as special sale information and user information.

[0634] An "API" is an interface that allows different software components to interact with each other.

[0635] This invention is a system that automates everything from generating menus to purchasing and delivering ingredients, which are necessary when a user plans a meal. Specific embodiments of this system are shown below.

[0636] Gathering information on special offers

[0637] The server uses a script that automatically runs at a specific time (for example, 3 AM every day) to retrieve special offer information from the APIs of multiple partner retailers. This ensures that the latest special offer information is always stored in the database. The server uses the Python requests library to send GET requests to each retailer's API endpoint and saves the returned JSON data to a database such as MySQL.

[0638] Entering user information

[0639] Users enter their allergy information and dietary conditions such as calorie restrictions through the application or website. This information is received by the device and sent to the server as a POST request in JSON format. When the user enters the necessary information in a form on the browser and clicks the "Submit" button, the front-end JavaScript code converts the data into JSON format and sends it to the server using the axios library.

[0640] User information storage

[0641] The server parses the user information it receives and stores it in a database. This makes it easy to reuse information that the user might need in the future. For example, the server parses JSON data received via a Flask or Django endpoint and saves it to a database such as MySQL.

[0642] Menu generation

[0643] The server generates a menu using a generative AI model (e.g., GPT-4) based on special offer information and user information. The server sends a prompt to the OpenAI API endpoint, and the AI ​​generates the menu based on that prompt. Examples of prompts include "Special offer information: chicken, curry powder, vegetables" and "User information: nut allergy, 1500 calorie limit." This generates a menu such as "chicken curry, salad, rice."

[0644] Menu

[0645] The generated menu is sent from the server to the user's device and displayed visually on the device. Users can view the menu and its details on devices such as PCs and smartphones. The server returns JSON data as a Flask or Django response, and the frontend displays the received data in HTML format.

[0646] Providing a list of ingredients

[0647] The server creates a list of necessary ingredients based on the generated menu and sends it to the user's terminal. An ingredient list such as "chicken, curry powder, various vegetables, rice" is displayed. The server selects the names of the necessary ingredients from the database to generate a list and sends it to the terminal in JSON format. The frontend receives the list data and displays it so that the user can visually confirm it.

[0648] Use of delivery service

[0649] When a user requests delivery, they make a selection through their device, and that information is sent to the server. The server sends the necessary information to the partner delivery service, and the groceries are delivered to the specified address. The server sends the "delivery address and list of required groceries" as a POST request in JSON format to the delivery company's API endpoint, and the actual delivery process is carried out.

[0650] This system allows users to plan economical and healthy meals without hassle and have ingredients delivered to their homes. This automation can reduce food waste and spoilage, and even improve health management.

[0651] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0652] Step 1: Gathering sale information

[0653] Processing flow:

[0654] The server automatically executes a script at a specific time (for example, 3 AM every day) to send GET requests to the APIs of multiple partner retailers. For example, the server accesses "https: / / api.storeA.com / sales" and "https: / / api.storeB.com / sales".

[0655] Specific actions:

[0656] The server sends a GET request using the Python requests library and receives JSON data returned as a response.

[0657] input:

[0658] The API endpoint URL for a specific retail store.

[0659] output:

[0660] Special sale information data in JSON format.

[0661] Data processing:

[0662] The server parses the received JSON data and executes an INSERT query to save it to the MySQL database.

[0663] Step 2: Enter user information

[0664] Processing flow:

[0665] Users enter dietary conditions such as allergy information and calorie restrictions through the application or website. For example, they might enter "I have a nut allergy" or "I need to limit myself to 1500 calories per day."

[0666] Specific actions:

[0667] The user enters information into a form in their browser and clicks the "Submit" button. JavaScript code on the frontend converts the input data into JSON format and sends a POST request to the server using the axios library.

[0668] input:

[0669] Allergy information and dietary conditions such as calorie restrictions entered by the user.

[0670] output:

[0671] User information data in JSON format.

[0672] Data processing:

[0673] The terminal parses the entered information in JSON format and sends it to the server.

[0674] Step 3: Saving User Information

[0675] Processing flow:

[0676] The server parses the received user information and stores it in the database. For example, it stores data corresponding to the fields "user_id", "allergies", and "calorie_limit".

[0677] Specific actions:

[0678] The server parses the JSON data received via the Flask or Django endpoint and saves it to the MySQL database using INSERT queries.

[0679] input:

[0680] User information data in JSON format.

[0681] output:

[0682] User information stored in the database.

[0683] Data processing:

[0684] The server parses the JSON data and inserts it into the corresponding database fields.

[0685] Step 4: Menu Generation

[0686] Processing flow:

[0687] The server sends prompts to a generating AI model (e.g., GPT-4) based on special offer information and user information, and the AI ​​generates a menu. For example, it might send prompts such as "Special offer information: Chicken, curry powder, vegetables" and "User information: Nut allergy, 1500 calorie limit".

[0688] Specific actions:

[0689] The server sends prompts to the OpenAI API endpoint using Python code and receives responses from the AI.

[0690] input:

[0691] A prompt message containing special offer information and user information.

[0692] output:

[0693] The generated menu data.

[0694] Data processing:

[0695] The server generates a prompt and sends it to the AI ​​model. The AI's response is analyzed and saved in JSON format.

[0696] Step 5: Serving the menu

[0697] Processing flow:

[0698] The generated menu is sent from the server to the user's device and displayed visually on the device. Users can check the details of the menu on devices such as PCs and smartphones.

[0699] Specific actions:

[0700] The server returns JSON data as a response from Flask or Django, which is then parsed on the frontend and displayed in HTML format.

[0701] input:

[0702] The generated menu data.

[0703] output:

[0704] Menu information displayed on the user's terminal.

[0705] Data processing:

[0706] The device parses the JSON data and generates HTML for visual display.

[0707] Step 6: Provide the ingredient list

[0708] Processing flow:

[0709] The server creates a list of necessary ingredients based on the generated menu and sends it to the user's terminal. For example, it might generate a list such as "chicken, curry powder, various vegetables, rice."

[0710] Specific actions:

[0711] The server selects the necessary ingredient information from the database, generates a list, and sends it to the terminal in JSON format.

[0712] input:

[0713] The generated menu data.

[0714] output:

[0715] A list of ingredients displayed on the user's terminal.

[0716] Data processing:

[0717] The server retrieves ingredient information from the database, creates a list, and sends it.

[0718] Step 7: Use a delivery service

[0719] Processing flow:

[0720] When a user requests grocery delivery, they send their selection information to a server via their device. The server then transmits the necessary information to a partner delivery service, and the groceries are delivered to the specified address.

[0721] Specific actions:

[0722] The user clicks the "Request Delivery" button in the application's UI, and the frontend sends a POST request to the server. The server then sends the delivery information to the delivery company's API endpoint via a POST request.

[0723] input:

[0724] User delivery preference information.

[0725] output:

[0726] Delivery information sent to the courier company.

[0727] Data processing:

[0728] The server analyzes the user's delivery information and sends it to the delivery company's API in the appropriate format.

[0729] (Application Example 1)

[0730] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0731] Traditional systems required users to manually search for sale information and create meal plans considering allergy information and calorie restrictions. Furthermore, it was difficult to effectively utilize sale information to generate economical meal plans, and the procedures for using food delivery services were cumbersome. Therefore, there was a need to simultaneously achieve effective use of sale information and health management.

[0732] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0733] In this invention, the server includes means for acquiring the latest special offer information, means for acquiring allergy information and dietary conditions such as calorie restrictions entered by the user, means for utilizing a generative AI model that generates menus based on the special offer information and the dietary conditions, means for creating a list of necessary ingredients based on the generated menus, and means for placing orders and having the ingredients delivered based on the ingredient list. As a result, users can effectively utilize special offer information, automatically generate menus suited to their individual dietary conditions, and easily purchase and have the necessary ingredients delivered.

[0734] "Means of obtaining the latest sale information" refers to a system or method that can regularly collect and store sale information provided by supermarkets and other retailers.

[0735] "Means for obtaining user-entered allergy information and dietary conditions such as calorie restrictions" refers to an interface for users to input individual dietary restrictions, such as allergies to specific foods or calorie restrictions, and a system for processing that information.

[0736] "Means of using a generative AI model that generates menus based on the aforementioned special offer information and the aforementioned meal conditions" refers to a system or method that uses an artificial intelligence model that automatically generates appropriate menus based on collected special offer information and the user's meal conditions as input.

[0737] "Means for creating a list of necessary ingredients based on the generated menu" refers to a system or method that has the function of compiling a list of necessary ingredients based on the generated menu.

[0738] "Means of ordering and delivering ingredients based on the aforementioned ingredient list" refers to a system or method that allows a user to order ingredients online based on a generated ingredient list and have those ingredients delivered to a specified address.

[0739] This invention is a system that allows users to plan healthy and economical meals without hassle and efficiently obtain the necessary ingredients. The system consists of a server, a user terminal, and partner retailers and delivery services.

[0740] First, the server periodically retrieves special offer information from multiple partner retailers. This information is collected via APIs provided by the retailers and stored in the server's database. The special offer information is retrieved in JSON format, and the database always maintains the most up-to-date information.

[0741] Next, the user enters allergy information and dietary conditions such as calorie restrictions into a dedicated application via their smartphone or computer. This information is sent from the device to a server and stored in a database.

[0742] The server uses a generative AI model to generate a menu tailored to the user, based on saved sale information and the user's dietary conditions. In this case, the generative AI model uses the sale information and user information as input. For example, based on user information such as "nut allergy, 1500 calories per day" and sale items such as "chicken, curry powder, vegetables," the model suggests a menu such as "chicken curry, salad, rice."

[0743] Furthermore, the server creates a list of necessary ingredients based on the generated menu and sends this list to the user's device. This list is displayed in a format that can be visually confirmed on the user's device.

[0744] When a user wants groceries delivered, they place an order through their device. This information is sent to the server, which uses the API of a partnered delivery service to arrange for the groceries to be delivered to the specified address.

[0745] For example, a menu is generated based on the user's input conditions, such as "nut allergy, 1500 calories per day," and special offer information, and a list of corresponding ingredients is displayed on the terminal. If the user selects ingredient delivery, the server automatically places an order with the delivery service, and the ingredients are delivered to the user's address.

[0746] An example of a prompt statement is as follows:

[0747] "User information: Has a nut allergy, daily calorie intake of 1500 kcal. Special offer information: Chicken, curry powder, vegetables."

[0748] This system allows users to efficiently plan healthy meals using sale information and easily obtain the necessary ingredients. As a result, it reduces the effort required from users, facilitates the effective use of sale information, improves health management, and reduces food waste.

[0749] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0750] Step 1:

[0751] The server periodically retrieves special offer information from the APIs of partner retailers. Specifically, the server sends a GET request to a designated URL of a retailer at a fixed time each day and receives the special offer information in JSON format. This JSON data is stored in a database. The input data is the special offer information retrieved from the retailer, and the output data is the special offer information stored in the database.

[0752] Step 2:

[0753] Users input their allergy information and dietary conditions, such as calorie restrictions, through a smartphone or computer application. This information is sent from the device to the server and stored in a database in JSON format. The input data is the user's dietary conditions, and the output data is the user's dietary conditions stored in the database.

[0754] Step 3:

[0755] The server sends special offer information retrieved from the database and the user's dietary conditions as input to the generating AI model. At this time, it generates a prompt message and makes a request to the AI ​​model. For example, it sends data in the format: "User information: Nut allergy, 1500 calories per day. Special offer information: Chicken, curry powder, vegetables." The generating AI model proposes an appropriate menu, which the server receives. The input data consists of the user's dietary conditions and special offer information, while the output data is the generated menu.

[0756] Step 4:

[0757] The server creates a list of necessary ingredients based on the generated menu. This list is generated in JSON format and sent to the user's terminal. Specifically, it analyzes the menu data obtained from the generation AI model and lists the necessary ingredients. The input data is the generated menu, and the output data is the ingredient list.

[0758] Step 5:

[0759] The user reviews the displayed list of ingredients and selects their delivery preferences through the application. This information is sent from the terminal to the server. The input data is the user's delivery preference information, and the output data is the delivery preference information stored on the server.

[0760] Step 6:

[0761] The server places an order with a partner delivery service via its API, based on the user's delivery preferences and ingredient list. Specifically, it converts the delivery address and required ingredient list into an API request format and sends it as a POST request. The delivery service then returns confirmation that the order has been received. The input data consists of the delivery preferences and ingredient list, while the output data is the order confirmation sent to the delivery service.

[0762] Through these steps, users can easily take advantage of special offers, create healthy and economical meal plans, and efficiently obtain the necessary ingredients.

[0763] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0764] This invention is a system that targets people who cook at home and generates economical and healthy menus that take into account supermarket sale information and the user's dietary conditions (allergy information and calorie restrictions). Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to suggest menus that are tailored to the user's mental state. In addition, it features integration with a delivery service, making it possible to deliver ingredients to people who have difficulty going shopping, such as those raising children or the elderly.

[0765] Program processing flow

[0766] Emotion recognition by an emotion engine

[0767] This system includes an emotion engine that recognizes the user's emotions. The emotion engine analyzes facial recognition data and voice data acquired from the user's device to determine the user's emotional state in real time.

[0768] Gathering information on special offers

[0769] The server retrieves special offer information from APIs of multiple partner retailers. This information is updated daily and stored in a database. The server sends requests to each retailer's API endpoint, parses the JSON data received as a response, and stores it in the database.

[0770] Entering and saving user information

[0771] Users enter allergy information, calorie restrictions, current emotional state, etc., through a website or application. The device collects this information and sends it to the server. The server stores the received information in a database.

[0772] Menu generation

[0773] Based on the special offer information acquired by the server, the user's dietary preferences, and the user's emotional state recognized by the emotion engine, a generative AI is used to generate a menu. For example, if the user is feeling stressed, a menu using ingredients with relaxing effects will be suggested.

[0774] Menu provision and adjustment

[0775] The generated menu is sent from the server to the user's terminal and displayed on the terminal. Users can check menus that accommodate allergies and calorie restrictions. Menus based on the user's emotional state are also suggested.

[0776] Generating an ingredient list

[0777] The server creates a list of necessary ingredients based on the generated menu and sends it to the user's terminal. The user can then review the list and proceed with placing an order.

[0778] Use of delivery service

[0779] When a user selects a delivery service, the device notifies the server of that information. The server then sends a list of necessary ingredients and the delivery address to the API of the partnered delivery service, and initiates the process of delivering the ingredients to the user's address.

[0780] Specific example

[0781] 1. Emotion recognition and acquisition of special offer information:

[0782] The user uses the device's camera and microphone to recognize their emotions. The emotion engine analyzes the user's facial recognition data and voice data and recognizes it as "stress."

[0783] The server retrieves the latest sale information from "Store A" and "Store B," and items such as "chicken," "curry powder," and "vegetables" are listed as sale items.

[0784] 2. Entering and saving user information:

[0785] The user enters "I have a nut allergy" and "I'm limiting myself to 1500 calories a day" into the application.

[0786] The server saves this information to the database.

[0787] 3. Menu generation:

[0788] Based on sale information, the user's meal preferences, and the results of the emotion engine, the server uses generative AI to generate a menu of "relaxing chicken soup, salad, and rice."

[0789] 4. Menu provision and adjustment:

[0790] The server sends the generated menu to the user's terminal, and the user checks the menu ("chicken soup, salad, rice") and the explanation of its relaxing effects in the application.

[0791] 5. Generating the ingredient list:

[0792] Based on the menu, a list of ingredients such as "chicken, curry powder, various vegetables, and rice" is displayed on the user's device.

[0793] 6. Use of delivery services:

[0794] The user selects a grocery delivery service, and the server sends the necessary information to the partnered delivery service.

[0795] A delivery service delivers groceries to the user's address.

[0796] As described above, this system comprehensively considers the user's current emotional state, health condition, and dietary conditions, and provides specific and practical menus and ingredients. As a result, users can lead a healthy lifestyle both physically and mentally.

[0797] The following describes the processing flow.

[0798] Step 1: Gathering sale information

[0799] The server uses a script that runs periodically to retrieve special offer information from the APIs of multiple partner retailers. Specifically, the server sends GET requests to each API endpoint at 3 AM every day and parses the JSON data received as a response.

[0800] The server saves the received sale information to a database. This ensures that the database always contains the most up-to-date sale information.

[0801] Step 2: Enter user information

[0802] The user logs in to the application or website.

[0803] The device displays an input form to the user, collecting allergy information, calorie restrictions, and other dietary conditions. For example, the user might enter information such as "I have a nut allergy" or "I have a 1500 calorie limit per day."

[0804] Review the information collected by the user and click the submit button.

[0805] Step 3: Submit User Information

[0806] The device collects user information and sends it to the server as a POST request in JSON format.

[0807] For example, the device sends the following data to the server.

[0808] json

[0809] {

[0810] "user_id": "12345",

[0811] "allergies": ["nuts"],

[0812] "calorie_limit": 1500

[0813] }

[0814] Step 4: Saving User Information

[0815] The server stores the received user information in a database. Allergy information and calorie restrictions are associated based on the user ID.

[0816] Step 5: Emotion recognition by the emotion engine

[0817] The device activates its function to acquire the user's facial recognition data and voice data.

[0818] The user shows their face to the device's camera and microphone and speaks.

[0819] The device sends facial recognition data and voice data to the emotion engine, which then analyzes it.

[0820] The server receives the analysis results and determines the user's current emotional state. For example, it might determine that the user is "stressed."

[0821] Step 6: Submit a menu generation request

[0822] The server sends a request to the generative AI based on the user's meal conditions, the results of the emotion engine, and special offer information.

[0823] Request data includes allergy information, calorie restrictions, a list of sale items, and the user's current emotional state.

[0824] json

[0825] {

[0826] "allergies": ["nuts"],

[0827] "calorie_limit": 1500,

[0828] "specials": ["chicken", "curry powder", "vegetables"],

[0829] "emotion": "stress"

[0830] }

[0831] Step 7: Menu generation using generative AI

[0832] The generative AI analyzes the received request data and generates a menu that is appropriate for the user's emotional state.

[0833] For example, if the emotion engine's result is "stress," it will generate a menu of "chicken soup, salad, and rice" which have a relaxing effect.

[0834] Step 8: Serving the menu

[0835] The server receives the generated menu data and sends it to the user's terminal.

[0836] The device displays the menu to the user, who can then view the details. For example, a menu such as "chicken soup, salad, and rice" and a description of its relaxing effects might be displayed.

[0837] Step 9: Generating the ingredient list

[0838] The server creates a list of necessary ingredients based on the generated menu.

[0839] The server sends a list of ingredients to the user's device. For example, an ingredient list such as "chicken, curry powder, various vegetables, and rice" will be displayed.

[0840] Step 10: Choosing a delivery service

[0841] The user selects a grocery delivery service.

[0842] The terminal notifies the server of its selection.

[0843] Step 11: Processing Orders and Deliveries

[0844] The server sends an order request, including a list of necessary ingredients and the delivery address, to the API of a partnered delivery service.

[0845] json

[0846] {

[0847] "address": "User's address",

[0848] "items": ["Chicken", "Curry powder", "Vegetables", "Rice"]

[0849] }

[0850] A delivery service receives the request and delivers the ingredients to the user's address.

[0851] This series of steps allows users to obtain economical and healthy meal plans and necessary ingredients without requiring specialized knowledge or time. Furthermore, meal suggestions that take into account the user's emotional state can comprehensively support their physical and mental health.

[0852] (Example 2)

[0853] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0854] Conventional menu generation systems only considered the user's dietary conditions and sale information, but did not take into account the user's emotional state. Therefore, they could not suggest menus that suited the user's mental state and could not fully meet their needs. Furthermore, few systems integrated grocery delivery services for users who could not go shopping. As a result, users were not receiving significant benefits.

[0855] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0856] In this invention, the server includes means for acquiring the latest sale information, means for acquiring allergy information and dietary conditions such as calorie restrictions entered by the user, means for recognizing the user's emotional state, means for using a generative AI model that generates a menu based on the sale information, the dietary conditions, and the emotional state, means for creating a list of necessary ingredients based on the generated menu, and means for placing an order based on the ingredient list and having the ingredients delivered. This enables the integration of menu suggestions tailored to the user's mental state and an ingredient delivery service.

[0857] 1. "Latest special offer information" refers to the latest sales information, such as prices and special offers, obtained from our partner retailers.

[0858] 2. "Allergy information" refers to information about a user's allergic reactions to specific foods.

[0859] 3. "Calorie restriction" refers to information about the limit on the total amount of calories a user can consume in a day.

[0860] 4. "Emotional state" refers to the psychological state determined by analyzing the user's facial recognition data and voice data.

[0861] 5. A "generative AI model" is an artificial intelligence model that generates a desired output when specific conditions are input.

[0862] 6. A "menu" is a specific meal plan generated based on the user's eating conditions and emotional state.

[0863] 7. A "food ingredient list" is a list containing the types and quantities of ingredients needed for the generated menu.

[0864] 8. A "delivery service" is a service in which a user orders selected ingredients and has them delivered to a specified address.

[0865] 9. A "terminal" is an electronic device (e.g., a smartphone, tablet, or personal computer) used by a user to input information or check results.

[0866] 10. A "server" is a central computer that manages and processes data for the entire system and provides various services.

[0867] This invention is a system that generates economical and healthy menus for users who cook at home, taking into account supermarket sale information and the user's dietary conditions (allergy information and calorie restrictions). Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to suggest menus that are tailored to the user's mental state. In addition, it features integration with a delivery service, enabling the delivery of ingredients to people who have difficulty going shopping, such as those raising children or the elderly.

[0868] This system includes the following main components: servers, terminals, and the hardware and software used by users.

[0869] Hardware and software to be used

[0870] 1. Server:

[0871] The server retrieves special offer information from APIs of multiple retailers and stores it in a database. It also generates meal plans using a generative AI model based on the user's dietary conditions and emotional state.

[0872] Software used: API clients, database management systems, generative AI models, etc.

[0873] 2. Terminal:

[0874] The terminal sends user data input to a server and displays the generated menu and ingredient list. It also uses a camera and microphone for emotion recognition.

[0875] Hardware to use: Smartphones, tablets, and personal computers.

[0876] Software used: emotion recognition engine, web browser or application, etc.

[0877] 3. User:

[0878] Users input allergy information and calorie restrictions via their device, and their emotional state is recognized. They also review suggested menus and ingredient lists, and select a delivery service as needed.

[0879] Specific examples of program processing

[0880] 1. Emotion recognition and acquisition of special offer information:

[0881] The user uses their device's camera and microphone to perform emotion recognition. The emotion engine determines the user's emotions in real time and recognizes them as, for example, "stress."

[0882] The server retrieves the latest sale information from partner retailers, and items such as "chicken," "curry powder," and "vegetables" are listed as sale items.

[0883] 2. Entering and saving user information:

[0884] Users enter information such as "I have a nut allergy" or "I have a 1500 calorie limit per day" through the application.

[0885] The device collects this information and sends it to the server.

[0886] The server saves the received information to the database.

[0887] 3. Menu generation:

[0888] The server generates prompt messages for the generative AI based on sale information, the user's dietary conditions, and the results of the emotion engine. Example of a prompt message: "Suggest a menu that the user is stressed. Sale information includes chicken, vegetables, and bread, and the user has a nut allergy."

[0889] The server inputs prompt text into a generative AI model, and a menu is generated, for example, "relaxing chicken soup, salad, and rice."

[0890] 4. Menu provision and adjustment:

[0891] The server sends the generated menu to the user's terminal.

[0892] The user reviews the suggested menu on their device and makes adjustments as needed.

[0893] 5. Generating the ingredient list:

[0894] The server creates a list of necessary ingredients based on the generated menu.

[0895] The server sends the ingredient list to the user's terminal.

[0896] 6. Use of delivery services:

[0897] The user selects a grocery delivery service through the app.

[0898] The device notifies the server of that information.

[0899] The server sends a list of necessary ingredients and the delivery address to the API of the partnered delivery service, and then processes the delivery.

[0900] Through the processing steps described above, this system comprehensively considers the user's current emotional state, health condition, and dietary requirements, and provides specific and practical menus and ingredients. As a result, the user can lead a healthy lifestyle both physically and mentally.

[0901] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0902] Step 1: Emotion Recognition

[0903] The user uses the device's camera and microphone to allow the system to recognize their emotional state. The device acquires facial recognition data and voice data in real time. This input data includes the user's facial expressions and voice tone.

[0904] The device sends this data to the emotion engine, which then performs analysis. For example, by analyzing the user's facial muscle movements and voice pitch, the emotion engine determines the user's emotional state as "stressed," "happy," or "tired."

[0905] Step 2: Gathering information on special offers

[0906] The server sends requests to the APIs of partner retailers to retrieve the latest sale information. This input data consists of the API endpoints of each retailer.

[0907] The server receives special offer information (e.g., "Chicken: 200 yen / 100g", "Curry powder: 100 yen / bag", "Vegetables: 150 yen / bag") in JSON format, analyzes it, and then stores the analysis results in the database.

[0908] Step 3: Enter and save user information

[0909] Users input their allergy information and dietary restrictions, such as calorie limits, through a website or application. Specific input information includes "I have a nut allergy" and "I have a 1500 calorie limit per day."

[0910] The device collects this information and sends it to the server.

[0911] The server saves the received user information to the database. This data processing stores the user's individual conditions in the database.

[0912] Step 4: Menu Generation

[0913] The server retrieves special offer information, user dietary conditions, and emotion engine results from the database. Based on this information, the generative AI generates prompt messages. The input data includes special offer information, allergy information, calorie restrictions, and emotional state.

[0914] Example of a prompt message: "When the user is feeling stressed, suggest a recommended meal. Special offers include chicken, vegetables, and bread, and the user has a nut allergy."

[0915] The server inputs a prompt message into the generative AI model and receives the generated menu. For example, a menu such as "relaxing chicken soup, salad, and rice" might be output.

[0916] Step 5: Menu provision and adjustment

[0917] The server sends the generated menu to the user's terminal. The output data is the generated menu.

[0918] The user reviews the suggested menu on their device and makes adjustments as needed. For example, the user reviews a menu of "chicken soup, salad, and rice" and its explanation of its relaxing effects.

[0919] Step 6: Generating the ingredient list

[0920] The server creates a list of necessary ingredients based on the generated menu. The input data is the generated menu.

[0921] The server sends a list of ingredients it has created to the user's terminal. The ingredient list includes "chicken, curry powder, various vegetables, and rice."

[0922] Step 7: Use a delivery service

[0923] Users select grocery delivery through the application.

[0924] The device notifies the server of that information.

[0925] The server sends the necessary information (food list and delivery address) to the API of the partnered delivery service, and the delivery process is initiated. Specifically, the server sends the food list to the API, and the package is delivered to the user's address.

[0926] In this way, the system comprehensively considers the user's emotional state, dietary conditions, and special offer information to provide the optimal menu, and further supports the easy acquisition of ingredients through home delivery services.

[0927] (Application Example 2)

[0928] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0929] Existing home cooking support systems offered menu suggestions that took into account users' allergy information and dietary restrictions such as calorie limits, but they lacked the functionality to consider the user's emotional state and suggest menus that responded to it in real time. Furthermore, there is a need to make shopping at physical stores more comfortable and effective by efficiently utilizing sale items and suggesting economical and healthy menus.

[0930] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring the latest sale information, means for acquiring allergy information and dietary conditions such as calorie restrictions entered by the user, emotion recognition means for recognizing the user's emotions, means for generating a menu based on the sale information, the dietary conditions, and the emotion recognition results, means for creating a list of necessary ingredients based on the generated menu, means for placing an order based on the ingredient list and having the ingredients delivered, and means for displaying the generated menu on the user's terminal. This enables menu suggestions that take into account the user's current emotional state and efficient shopping at physical stores.

[0931] "Latest special offer information" refers to special price information for currently available products, obtained from multiple partner retailers.

[0932] "User-entered allergy information" refers to information about a user's allergies that they provide to the system.

[0933] "Calorie restriction" refers to the upper limit of daily calorie intake set by the user.

[0934] "Dietary conditions" refer to specific conditions or restrictions related to diet, such as the user's allergy information or calorie restrictions.

[0935] "Emotion recognition technology" refers to technology that analyzes and recognizes a user's emotions in real time.

[0936] The "means for generating menus" refer to a function that suggests appropriate meal menus based on acquired special offer information, meal conditions, and emotion recognition results.

[0937] The "list of necessary ingredients" is a list of all the ingredients needed to actually prepare the generated menu.

[0938] "A means of placing an order and having ingredients delivered" refers to a function that allows users to purchase ingredients they specify and arrange for them to be delivered to a designated address.

[0939] "User's device" refers to a smartphone, smart glasses, or any other electronic device used by the user.

[0940] A "generative AI model" is an artificial intelligence model that automatically generates menus tailored to user needs based on diverse data.

[0941] To implement this invention, a server, user terminals (smartphones and smart glasses), and API connections from multiple retailers are utilized. The specific hardware and software, data processing, and data calculations required for this are described below.

[0942] Hardware and software

[0943] Server: A computer server installed in a cloud environment.

[0944] User devices: Smartphones, smart glasses (e.g., Google Glass)

[0945] Emotion recognition engine: Microsoft Azure Emotion API

[0946] Database management system: MySQL

[0947] Special Sale Information Acquisition API: APIs for various supermarkets

[0948] Generative AI: OpenAI GPT-3

[0949] Data processing and data calculation

[0950] Emotion recognition and acquisition of sale information

[0951] The device's camera and microphone are used to send user facial recognition data and voice data to an emotion recognition engine. The emotion recognition engine analyzes the user's emotions and displays the results on the device in real time. The server retrieves the latest special offer information from the APIs of partner retailers, receives it in JSON format, and stores it in a database.

[0952] Entering and saving user information

[0953] Upon initial login, users enter allergy information and calorie restrictions via their device. This information is sent to the server and stored in the database.

[0954] Menu generation

[0955] The server retrieves special offer information, user health information, and emotion recognition results from the database. Then, it inputs prompts like the following into the generative AI to generate a menu.

[0956] Example: Prompt text to input to a generative AI

[0957] The user has a dairy allergy and a daily calorie restriction of 1800 calories. The user's current emotional state is fatigue. Please suggest a meal consisting of stir-fried chicken and broccoli, a salad, and whole-wheat bread, all of which are known to help with fatigue recovery.

[0958] The generated menu is saved to a database by the server.

[0959] Menu provision and adjustment

[0960] Users can view the generated menu by wearing smart glasses or using a smartphone app in the store. They can adjust the menu as needed.

[0961] Generating an ingredient list

[0962] Based on the proposed menu, the server extracts the necessary ingredients from the database and displays them on the user's terminal.

[0963] Use of delivery service

[0964] When a user selects a delivery service, the server retrieves the necessary information from the database and sends it to the API of the partnered delivery service. The delivery service then delivers the specified ingredients to the user's address.

[0965] As a concrete example, if a user wears smart glasses and the emotion recognition engine detects "fatigue," the server retrieves "chicken," "broccoli," and "olive oil" as sale items from "Supermarket A." Based on the information the user has entered, such as "dairy allergy" and "1800 calorie limit per day," the generative AI generates a menu of "stir-fried chicken and broccoli, salad, and whole wheat bread." The user confirms this menu, and "chicken, broccoli, olive oil, and whole wheat bread" are displayed as necessary ingredients. If the user selects a delivery service, the ingredients are delivered to their home.

[0966] As a result, users can receive menu suggestions that take their emotional state into consideration and achieve efficient shopping at physical stores.

[0967] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0968] Step 1:

[0969] Users log in to the system via smart glasses or smartphones. After logging in, they enter initial settings such as allergy information and dietary conditions including calorie restrictions. The entered information is sent from the device to the server and stored in the database.

[0970] Input: User allergy information, calorie restriction settings

[0971] Data processing: Information collection and format conversion

[0972] Output: User information stored in the database

[0973] Step 2:

[0974] The user uses the smart glasses' camera and microphone to send facial recognition data and voice data to the emotion recognition engine. The emotion recognition engine analyzes this data and recognizes the user's emotional state in real time. The recognition results are displayed on the device.

[0975] Input: User's facial recognition data, voice data

[0976] Data processing: Data analysis using an emotion recognition engine.

[0977] Output: Real-time emotional state

[0978] Step 3:

[0979] The server retrieves the latest sale information from the APIs of each partner retailer. The retrieved sale information is received in JSON format, the server analyzes the data, and saves the necessary information to the database.

[0980] Input: Special offer information from retailer API

[0981] Data processing: Parsing JSON data and storing it in a database.

[0982] Output: Special sale information stored in the database

[0983] Step 4:

[0984] The server retrieves the user's dietary conditions (allergy information, calorie restrictions), emotion recognition results, and special offer information from the database. Based on this information, it generates prompt messages, which are then input into a generative AI to generate a menu.

[0985] Input: User's dietary conditions, emotional state, special offer information

[0986] Data processing: Prompt processing by generative AI

[0987] Output: Generated menu

[0988] Examples of prompt statements:

[0989] The user has a dairy allergy and a daily calorie restriction of 1800 calories. The user's current emotional state is fatigue. Please suggest a meal consisting of stir-fried chicken and broccoli, a salad, and whole-wheat bread, all of which are known to help with fatigue recovery.

[0990] Step 5:

[0991] The server sends the generated menu to the user's terminal. The user can review the menu and make adjustments as needed. The adjustment results are then sent back to the server.

[0992] Input: Generated menu

[0993] Data processing: Information transmission and user adjustments.

[0994] Output: Adjusted final menu

[0995] Step 6:

[0996] The server creates a list of necessary ingredients based on the final menu and sends it to the terminal. The user reviews the list and selects the ingredients they want to order.

[0997] Input: Final Menu

[0998] Data processing: Extraction and creation of a list of necessary ingredients.

[0999] Output: Ingredient list

[1000] Step 7:

[1001] When a user selects grocery delivery, the device notifies the server of this information. The server then sends the necessary information to the API of its partner delivery service and initiates the process of delivering the groceries to the user's address.

[1002] Input: Ingredient list, delivery selection information

[1003] Data processing: Sending data to delivery service APIs

[1004] Output: Food items delivered to the user's address

[1005] The above outlines the specific processing steps of the program of this invention. Through these steps, users can achieve healthier and more efficient shopping based on personalized suggestions that take their emotional state into consideration.

[1006] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1007] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1008] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1009] [Third Embodiment]

[1010] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1011] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1012] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1013] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1014] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1015] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1016] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1017] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1018] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1019] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1020] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1021] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1022] This invention is a system that targets people who cook at home and generates economical and healthy menus by taking into account supermarket sale information and dietary conditions specified by the user (allergy information and calorie restrictions). Furthermore, by linking with delivery services, it is possible to deliver ingredients to people who have difficulty going shopping, such as those raising children or the elderly.

[1023] This system mainly consists of the following elements:

[1024] 1. Gathering information on special offers

[1025] 2. Entering and saving user information

[1026] 3. Menu generation

[1027] 4. Providing a list of ingredients

[1028] 5. Use of delivery services

[1029] Program processing flow

[1030] Gathering information on special offers

[1031] The server uses a script that runs at specific times to retrieve special offer information from the APIs of multiple partner retailers. For example, the server sends a GET request to a specific URL at 3 AM every day and saves the received JSON data to the database. In this way, the latest special offer information is always stored in the database.

[1032] Entering user information

[1033] Users input dietary information such as allergy details and calorie restrictions through applications or websites. The device receives this information and sends it to the server. For example, a user might enter information such as "I have a nut allergy" and "I have a 1500 calorie limit per day" into an input form. The device then sends this data to the server as a POST request in JSON format.

[1034] User information storage

[1035] The server stores the received user information in a database. This makes it easy to reuse information that the user may need in the future.

[1036] Menu generation

[1037] The server uses a generative AI to generate menus based on user information and acquired sale information. For example, the server sends the aforementioned user information and sale information to the AI, which then suggests a menu like "chicken curry, salad, and rice."

[1038] Menu

[1039] The generated menu is sent from the server to the user's device and displayed visually on the device. Users can check the menu and its details on devices such as PCs and smartphones.

[1040] Providing a list of ingredients

[1041] Based on the menu, the server creates a list of necessary ingredients and sends it to the user's terminal. For example, an ingredient list such as "chicken, curry powder, various vegetables, rice" will be displayed.

[1042] Use of delivery service

[1043] When a user requests delivery, they make a selection through their device, and that information is sent to the server. The server then sends the necessary information to the partner delivery service, and the groceries are delivered to the specified address. For example, the server might send the "delivery address and list of required groceries" as a POST request to the delivery company's API.

[1044] Specific example

[1045] In reality, the following scenario is possible:

[1046] 1. Gathering information on special offers:

[1047] The server retrieves special offer information from "Store A" and "Store B," and items such as "chicken," "curry powder," and "vegetables" are listed as special offer products.

[1048] 2. Entering and saving user information:

[1049] The user accesses the website and enters "nut allergy" as allergy information and "1500 calories" as calorie restriction.

[1050] 3. Menu generation:

[1051] Based on the aforementioned special offer information and user information, the server uses a generative AI to generate a menu consisting of "chicken curry, salad, and rice."

[1052] 4. Menu provision:

[1053] The server sends this menu to the user's device, and the user checks the menu on their smartphone.

[1054] 5. Provide a list of ingredients:

[1055] Based on the menu, a list of ingredients such as "chicken, curry powder, various vegetables, and rice" will be displayed on the user's device.

[1056] 6. Use of delivery services:

[1057] The user selects a grocery delivery service, and the server initiates the process of delivering the groceries to the specified address.

[1058] This system allows users to plan economical and healthy meals without hassle and have ingredients delivered to their homes. This helps reduce food waste and loss, and also allows for better health management.

[1059] The following describes the processing flow.

[1060] Step 1: Gathering sale information

[1061] The server uses a script that runs periodically to retrieve special offer information from the APIs of partner retailers.

[1062] The server sends GET requests to each retailer's API endpoint and parses the JSON data received as a response.

[1063] The server analyzes the sale information and saves it to the database. This ensures that the latest sale information is always available.

[1064] Step 2: Enter user information

[1065] The user logs in to the application or website.

[1066] The device displays an input form to the user and collects allergy information, calorie restrictions, and other dietary conditions.

[1067] The user enters this information and clicks the submit button.

[1068] Step 3: Submit User Information

[1069] The device collects user information and sends it to the server as a POST request in JSON format.

[1070] For example, the device sends the following data to the server.

[1071] json

[1072] {

[1073] "user_id": "12345",

[1074] "allergies": ["nuts"],

[1075] "calorie_limit": 1500

[1076] }

[1077] Step 4: Saving User Information

[1078] The server stores the received user information in the database.

[1079] The server associates allergy information and calorie restrictions based on the user ID.

[1080] Step 5: Submit a menu generation request

[1081] The server sends a request to the generative AI based on the user's dietary requirements and the latest special offers.

[1082] Request data includes allergy information, calorie restrictions, and a list of sale items.

[1083] json

[1084] {

[1085] "allergies": ["nuts"],

[1086] "calorie_limit": 1500,

[1087] "Specials": ["Chicken", "Curry powder", "Vegetables"]

[1088] }

[1089] Step 6: Menu generation using generative AI

[1090] The generative AI analyzes the received request data and generates the optimal menu.

[1091] As an example, generate a menu consisting of "chicken curry, salad, and rice."

[1092] Step 7: Serving the menu

[1093] The server receives the generated menu data and sends it to the user's terminal.

[1094] The device displays the generated menu to the user. The user can view the menu details and cooking instructions.

[1095] Step 8: Generating the ingredient list

[1096] The server creates a list of necessary ingredients based on the generated menu.

[1097] The server sends the ingredient list to the user's terminal.

[1098] json

[1099] {

[1100] "items": ["Chicken", "Curry powder", "Vegetables", "Rice"]

[1101] }

[1102] Step 9: Choosing a delivery service

[1103] The user selects a grocery delivery service on their device.

[1104] The terminal notifies the server of the user's selection.

[1105] Step 10: Processing Orders and Deliveries

[1106] The server sends an order request, including a list of necessary ingredients and the delivery address, to the API of a partnered delivery service.

[1107] json

[1108] {

[1109] "address": "User's address",

[1110] "items": ["Chicken", "Curry powder", "Vegetables", "Rice"]

[1111] }

[1112] A delivery service receives the request and delivers the ingredients to the user's address.

[1113] This series of steps allows users to automatically obtain economical and healthy meal plans and necessary ingredients, which are then delivered to their homes. This eliminates the hassle of shopping and reduces food waste and unnecessary expenses.

[1114] (Example 1)

[1115] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1116] With existing systems, users have to manually collect sale information and create meal plans while considering allergy information and calorie restrictions, which is time-consuming. This is even more difficult for people who have difficulty going grocery shopping, such as the elderly and those with young children. This can lead to wasted food, food loss, and difficulties in managing one's health.

[1117] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1118] In this invention, the server includes means for collecting the latest sale information, means for obtaining allergy information and dietary conditions such as calorie restrictions entered by the user, means for generating a menu using a generation AI model based on the sale information and dietary conditions, means for creating a list of necessary ingredients based on the generated menu, and means for placing an order based on the ingredient list and having the ingredients delivered. This makes it possible for users to obtain economical and healthy menus without any hassle and have the ingredients delivered to their homes.

[1119] "Special sale information" refers to information about discounts and special offers on products that can be obtained from partner retailers.

[1120] "User information" refers to information about dietary conditions such as allergy information and calorie restrictions that users enter.

[1121] A "generative AI model" is an artificial intelligence model used to generate menus based on special offer information and user information.

[1122] A "menu" refers to a combination of meals suggested by an AI model based on special offer information and user data.

[1123] A "food ingredient list" is a list of ingredients needed based on the generated menu.

[1124] "Delivery service" refers to the means of arranging for the delivery of groceries ordered by a user to a specified address.

[1125] A "terminal" is an electronic device used by users to input information or to display generated menus and ingredient lists.

[1126] The "server" is a central computer system that performs tasks such as collecting special offer information, storing user information, operating the generated AI model, creating menus and ingredient lists, and arranging delivery methods.

[1127] A "database" is a system for storing and managing data such as special sale information and user information.

[1128] An "API" is an interface that allows different software components to interact with each other.

[1129] This invention is a system that automates everything from generating menus to purchasing and delivering ingredients, which are necessary when a user plans a meal. Specific embodiments of this system are shown below.

[1130] Gathering information on special offers

[1131] The server uses a script that automatically runs at a specific time (for example, 3 AM every day) to retrieve special offer information from the APIs of multiple partner retailers. This ensures that the latest special offer information is always stored in the database. The server uses the Python requests library to send GET requests to each retailer's API endpoint and saves the returned JSON data to a database such as MySQL.

[1132] Entering user information

[1133] Users enter their allergy information and dietary conditions such as calorie restrictions through the application or website. This information is received by the device and sent to the server as a POST request in JSON format. When the user enters the necessary information in a form on the browser and clicks the "Submit" button, the front-end JavaScript code converts the data into JSON format and sends it to the server using the axios library.

[1134] User information storage

[1135] The server parses the user information it receives and stores it in a database. This makes it easy to reuse information that the user might need in the future. For example, the server parses JSON data received via a Flask or Django endpoint and saves it to a database such as MySQL.

[1136] Menu generation

[1137] The server generates a menu using a generative AI model (e.g., GPT-4) based on special offer information and user information. The server sends a prompt to the OpenAI API endpoint, and the AI ​​generates the menu based on that prompt. Examples of prompts include "Special offer information: chicken, curry powder, vegetables" and "User information: nut allergy, 1500 calorie limit." This generates a menu such as "chicken curry, salad, rice."

[1138] Menu

[1139] The generated menu is sent from the server to the user's device and displayed visually on the device. Users can view the menu and its details on devices such as PCs and smartphones. The server returns JSON data as a Flask or Django response, and the frontend displays the received data in HTML format.

[1140] Providing a list of ingredients

[1141] The server creates a list of necessary ingredients based on the generated menu and sends it to the user's terminal. An ingredient list such as "chicken, curry powder, various vegetables, rice" is displayed. The server selects the names of the necessary ingredients from the database to generate a list and sends it to the terminal in JSON format. The frontend receives the list data and displays it so that the user can visually confirm it.

[1142] Use of delivery service

[1143] When a user requests delivery, they make a selection through their device, and that information is sent to the server. The server sends the necessary information to the partner delivery service, and the groceries are delivered to the specified address. The server sends the "delivery address and list of required groceries" as a POST request in JSON format to the delivery company's API endpoint, and the actual delivery process is carried out.

[1144] This system allows users to plan economical and healthy meals without hassle and have ingredients delivered to their homes. This automation can reduce food waste and spoilage, and even improve health management.

[1145] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1146] Step 1: Gathering sale information

[1147] Processing flow:

[1148] The server automatically executes a script at a specific time (for example, 3 AM every day) to send GET requests to the APIs of multiple partner retailers. For example, the server accesses "https: / / api.storeA.com / sales" and "https: / / api.storeB.com / sales".

[1149] Specific actions:

[1150] The server sends a GET request using the Python requests library and receives JSON data returned as a response.

[1151] input:

[1152] The API endpoint URL for a specific retail store.

[1153] output:

[1154] Special sale information data in JSON format.

[1155] Data processing:

[1156] The server parses the received JSON data and executes an INSERT query to save it to the MySQL database.

[1157] Step 2: Enter user information

[1158] Processing flow:

[1159] Users enter dietary conditions such as allergy information and calorie restrictions through the application or website. For example, they might enter "I have a nut allergy" or "I need to limit myself to 1500 calories per day."

[1160] Specific actions:

[1161] The user enters information into a form in their browser and clicks the "Submit" button. JavaScript code on the frontend converts the input data into JSON format and sends a POST request to the server using the axios library.

[1162] input:

[1163] Allergy information and dietary conditions such as calorie restrictions entered by the user.

[1164] output:

[1165] User information data in JSON format.

[1166] Data processing:

[1167] The terminal parses the entered information in JSON format and sends it to the server.

[1168] Step 3: Saving User Information

[1169] Processing flow:

[1170] The server parses the received user information and stores it in the database. For example, it stores data corresponding to the fields "user_id", "allergies", and "calorie_limit".

[1171] Specific actions:

[1172] The server parses the JSON data received via the Flask or Django endpoint and saves it to the MySQL database using INSERT queries.

[1173] input:

[1174] User information data in JSON format.

[1175] output:

[1176] User information stored in the database.

[1177] Data processing:

[1178] The server parses the JSON data and inserts it into the corresponding database fields.

[1179] Step 4: Menu Generation

[1180] Processing flow:

[1181] The server sends prompts to a generating AI model (e.g., GPT-4) based on special offer information and user information, and the AI ​​generates a menu. For example, it might send prompts such as "Special offer information: Chicken, curry powder, vegetables" and "User information: Nut allergy, 1500 calorie limit".

[1182] Specific actions:

[1183] The server sends prompts to the OpenAI API endpoint using Python code and receives responses from the AI.

[1184] input:

[1185] A prompt message containing special offer information and user information.

[1186] output:

[1187] The generated menu data.

[1188] Data processing:

[1189] The server generates a prompt and sends it to the AI ​​model. The AI's response is analyzed and saved in JSON format.

[1190] Step 5: Serving the menu

[1191] Processing flow:

[1192] The generated menu is sent from the server to the user's device and displayed visually on the device. Users can check the details of the menu on devices such as PCs and smartphones.

[1193] Specific actions:

[1194] The server returns JSON data as a response from Flask or Django, which is then parsed on the frontend and displayed in HTML format.

[1195] input:

[1196] The generated menu data.

[1197] output:

[1198] Menu information displayed on the user's terminal.

[1199] Data processing:

[1200] The device parses the JSON data and generates HTML for visual display.

[1201] Step 6: Provide the ingredient list

[1202] Processing flow:

[1203] The server creates a list of necessary ingredients based on the generated menu and sends it to the user's terminal. For example, it might generate a list such as "chicken, curry powder, various vegetables, rice."

[1204] Specific actions:

[1205] The server selects the necessary ingredient information from the database, generates a list, and sends it to the terminal in JSON format.

[1206] input:

[1207] The generated menu data.

[1208] output:

[1209] A list of ingredients displayed on the user's terminal.

[1210] Data processing:

[1211] The server retrieves ingredient information from the database, creates a list, and sends it.

[1212] Step 7: Use a delivery service

[1213] Processing flow:

[1214] When a user requests grocery delivery, they send their selection information to a server via their device. The server then transmits the necessary information to a partner delivery service, and the groceries are delivered to the specified address.

[1215] Specific actions:

[1216] The user clicks the "Request Delivery" button in the application's UI, and the frontend sends a POST request to the server. The server then sends the delivery information to the delivery company's API endpoint via a POST request.

[1217] input:

[1218] User delivery preference information.

[1219] output:

[1220] Delivery information sent to the courier company.

[1221] Data processing:

[1222] The server analyzes the user's delivery information and sends it to the delivery company's API in the appropriate format.

[1223] (Application Example 1)

[1224] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1225] Traditional systems required users to manually search for sale information and create meal plans considering allergy information and calorie restrictions. Furthermore, it was difficult to effectively utilize sale information to generate economical meal plans, and the procedures for using food delivery services were cumbersome. Therefore, there was a need to simultaneously achieve effective use of sale information and health management.

[1226] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1227] In this invention, the server includes means for acquiring the latest special offer information, means for acquiring allergy information and dietary conditions such as calorie restrictions entered by the user, means for utilizing a generative AI model that generates menus based on the special offer information and the dietary conditions, means for creating a list of necessary ingredients based on the generated menus, and means for placing orders and having the ingredients delivered based on the ingredient list. As a result, users can effectively utilize special offer information, automatically generate menus suited to their individual dietary conditions, and easily purchase and have the necessary ingredients delivered.

[1228] "Means of obtaining the latest sale information" refers to a system or method that can regularly collect and store sale information provided by supermarkets and other retailers.

[1229] "Means for obtaining user-entered allergy information and dietary conditions such as calorie restrictions" refers to an interface for users to input individual dietary restrictions, such as allergies to specific foods or calorie restrictions, and a system for processing that information.

[1230] "Means of using a generative AI model that generates menus based on the aforementioned special offer information and the aforementioned meal conditions" refers to a system or method that uses an artificial intelligence model that automatically generates appropriate menus based on collected special offer information and the user's meal conditions as input.

[1231] "Means for creating a list of necessary ingredients based on the generated menu" refers to a system or method that has the function of compiling a list of necessary ingredients based on the generated menu.

[1232] "Means of ordering and delivering ingredients based on the aforementioned ingredient list" refers to a system or method that allows a user to order ingredients online based on a generated ingredient list and have those ingredients delivered to a specified address.

[1233] This invention is a system that allows users to plan healthy and economical meals without hassle and efficiently obtain the necessary ingredients. The system consists of a server, a user terminal, and partner retailers and delivery services.

[1234] First, the server periodically retrieves special offer information from multiple partner retailers. This information is collected via APIs provided by the retailers and stored in the server's database. The special offer information is retrieved in JSON format, and the database always maintains the most up-to-date information.

[1235] Next, the user enters allergy information and dietary conditions such as calorie restrictions into a dedicated application via their smartphone or computer. This information is sent from the device to a server and stored in a database.

[1236] The server uses a generative AI model to generate a menu tailored to the user, based on saved sale information and the user's dietary conditions. In this case, the generative AI model uses the sale information and user information as input. For example, based on user information such as "nut allergy, 1500 calories per day" and sale items such as "chicken, curry powder, vegetables," the model suggests a menu such as "chicken curry, salad, rice."

[1237] Furthermore, the server creates a list of necessary ingredients based on the generated menu and sends this list to the user's device. This list is displayed in a format that can be visually confirmed on the user's device.

[1238] When a user wants groceries delivered, they place an order through their device. This information is sent to the server, which uses the API of a partnered delivery service to arrange for the groceries to be delivered to the specified address.

[1239] For example, a menu is generated based on the user's input conditions, such as "nut allergy, 1500 calories per day," and special offer information, and a list of corresponding ingredients is displayed on the terminal. If the user selects ingredient delivery, the server automatically places an order with the delivery service, and the ingredients are delivered to the user's address.

[1240] An example of a prompt statement is as follows:

[1241] "User information: Has a nut allergy, daily calorie intake of 1500 kcal. Special offer information: Chicken, curry powder, vegetables."

[1242] This system allows users to efficiently plan healthy meals using sale information and easily obtain the necessary ingredients. As a result, it reduces the effort required from users, facilitates the effective use of sale information, improves health management, and reduces food waste.

[1243] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1244] Step 1:

[1245] The server periodically retrieves special offer information from the APIs of partner retailers. Specifically, the server sends a GET request to a designated URL of a retailer at a fixed time each day and receives the special offer information in JSON format. This JSON data is stored in a database. The input data is the special offer information retrieved from the retailer, and the output data is the special offer information stored in the database.

[1246] Step 2:

[1247] Users input their allergy information and dietary conditions, such as calorie restrictions, through a smartphone or computer application. This information is sent from the device to the server and stored in a database in JSON format. The input data is the user's dietary conditions, and the output data is the user's dietary conditions stored in the database.

[1248] Step 3:

[1249] The server sends special offer information retrieved from the database and the user's dietary conditions as input to the generating AI model. At this time, it generates a prompt message and makes a request to the AI ​​model. For example, it sends data in the format: "User information: Nut allergy, 1500 calories per day. Special offer information: Chicken, curry powder, vegetables." The generating AI model proposes an appropriate menu, which the server receives. The input data consists of the user's dietary conditions and special offer information, while the output data is the generated menu.

[1250] Step 4:

[1251] The server creates a list of necessary ingredients based on the generated menu. This list is generated in JSON format and sent to the user's terminal. Specifically, it analyzes the menu data obtained from the generation AI model and lists the necessary ingredients. The input data is the generated menu, and the output data is the ingredient list.

[1252] Step 5:

[1253] The user reviews the displayed list of ingredients and selects their delivery preferences through the application. This information is sent from the terminal to the server. The input data is the user's delivery preference information, and the output data is the delivery preference information stored on the server.

[1254] Step 6:

[1255] The server places an order with a partner delivery service via its API, based on the user's delivery preferences and ingredient list. Specifically, it converts the delivery address and required ingredient list into an API request format and sends it as a POST request. The delivery service then returns confirmation that the order has been received. The input data consists of the delivery preferences and ingredient list, while the output data is the order confirmation sent to the delivery service.

[1256] Through these steps, users can easily take advantage of special offers, create healthy and economical meal plans, and efficiently obtain the necessary ingredients.

[1257] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1258] This invention is a system that targets people who cook at home and generates economical and healthy menus that take into account supermarket sale information and the user's dietary conditions (allergy information and calorie restrictions). Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to suggest menus that are tailored to the user's mental state. In addition, it features integration with a delivery service, making it possible to deliver ingredients to people who have difficulty going shopping, such as those raising children or the elderly.

[1259] Program processing flow

[1260] Emotion recognition by an emotion engine

[1261] This system includes an emotion engine that recognizes the user's emotions. The emotion engine analyzes facial recognition data and voice data acquired from the user's device to determine the user's emotional state in real time.

[1262] Gathering information on special offers

[1263] The server retrieves special offer information from APIs of multiple partner retailers. This information is updated daily and stored in a database. The server sends requests to each retailer's API endpoint, parses the JSON data received as a response, and stores it in the database.

[1264] Entering and saving user information

[1265] Users enter allergy information, calorie restrictions, current emotional state, etc., through a website or application. The device collects this information and sends it to the server. The server stores the received information in a database.

[1266] Menu generation

[1267] Based on the special offer information acquired by the server, the user's dietary preferences, and the user's emotional state recognized by the emotion engine, a generative AI is used to generate a menu. For example, if the user is feeling stressed, a menu using ingredients with relaxing effects will be suggested.

[1268] Menu provision and adjustment

[1269] The generated menu is sent from the server to the user's terminal and displayed on the terminal. Users can check menus that accommodate allergies and calorie restrictions. Menus based on the user's emotional state are also suggested.

[1270] Generating an ingredient list

[1271] The server creates a list of necessary ingredients based on the generated menu and sends it to the user's terminal. The user can then review the list and proceed with placing an order.

[1272] Use of delivery service

[1273] When a user selects a delivery service, the device notifies the server of that information. The server then sends a list of necessary ingredients and the delivery address to the API of the partnered delivery service, and initiates the process of delivering the ingredients to the user's address.

[1274] Specific example

[1275] 1. Emotion recognition and acquisition of special offer information:

[1276] The user uses the device's camera and microphone to recognize their emotions. The emotion engine analyzes the user's facial recognition data and voice data and recognizes it as "stress."

[1277] The server retrieves the latest sale information from "Store A" and "Store B," and items such as "chicken," "curry powder," and "vegetables" are listed as sale items.

[1278] 2. Entering and saving user information:

[1279] The user enters "I have a nut allergy" and "I'm limiting myself to 1500 calories a day" into the application.

[1280] The server saves this information to the database.

[1281] 3. Menu generation:

[1282] Based on sale information, the user's meal preferences, and the results of the emotion engine, the server uses generative AI to generate a menu of "relaxing chicken soup, salad, and rice."

[1283] 4. Menu provision and adjustment:

[1284] The server sends the generated menu to the user's terminal, and the user checks the menu ("chicken soup, salad, rice") and the explanation of its relaxing effects in the application.

[1285] 5. Generating the ingredient list:

[1286] Based on the menu, a list of ingredients such as "chicken, curry powder, various vegetables, and rice" is displayed on the user's device.

[1287] 6. Use of delivery services:

[1288] The user selects a grocery delivery service, and the server sends the necessary information to the partnered delivery service.

[1289] A delivery service delivers groceries to the user's address.

[1290] As described above, this system comprehensively considers the user's current emotional state, health condition, and dietary conditions, and provides specific and practical menus and ingredients. As a result, users can lead a healthy lifestyle both physically and mentally.

[1291] The following describes the processing flow.

[1292] Step 1: Gathering sale information

[1293] The server uses a script that runs periodically to retrieve special offer information from the APIs of multiple partner retailers. Specifically, the server sends GET requests to each API endpoint at 3 AM every day and parses the JSON data received as a response.

[1294] The server saves the received sale information to a database. This ensures that the database always contains the most up-to-date sale information.

[1295] Step 2: Enter user information

[1296] The user logs in to the application or website.

[1297] The device displays an input form to the user, collecting allergy information, calorie restrictions, and other dietary conditions. For example, the user might enter information such as "I have a nut allergy" or "I have a 1500 calorie limit per day."

[1298] Review the information collected by the user and click the submit button.

[1299] Step 3: Submit User Information

[1300] The device collects user information and sends it to the server as a POST request in JSON format.

[1301] For example, the device sends the following data to the server.

[1302] json

[1303] {

[1304] "user_id": "12345",

[1305] "allergies": ["nuts"],

[1306] "calorie_limit": 1500

[1307] }

[1308] Step 4: Saving User Information

[1309] The server stores the received user information in a database. Allergy information and calorie restrictions are associated based on the user ID.

[1310] Step 5: Emotion recognition by the emotion engine

[1311] The device activates its function to acquire the user's facial recognition data and voice data.

[1312] The user shows their face to the device's camera and microphone and speaks.

[1313] The device sends facial recognition data and voice data to the emotion engine, which then analyzes it.

[1314] The server receives the analysis results and determines the user's current emotional state. For example, it might determine that the user is "stressed."

[1315] Step 6: Submit a menu generation request

[1316] The server sends a request to the generative AI based on the user's meal conditions, the results of the emotion engine, and special offer information.

[1317] Request data includes allergy information, calorie restrictions, a list of sale items, and the user's current emotional state.

[1318] json

[1319] {

[1320] "allergies": ["nuts"],

[1321] "calorie_limit": 1500,

[1322] "specials": ["chicken", "curry powder", "vegetables"],

[1323] "emotion": "stress"

[1324] }

[1325] Step 7: Menu generation using generative AI

[1326] The generative AI analyzes the received request data and generates a menu that is appropriate for the user's emotional state.

[1327] For example, if the emotion engine's result is "stress," it will generate a menu of "chicken soup, salad, and rice" which have a relaxing effect.

[1328] Step 8: Serving the menu

[1329] The server receives the generated menu data and sends it to the user's terminal.

[1330] The device displays the menu to the user, who can then view the details. For example, a menu such as "chicken soup, salad, and rice" and a description of its relaxing effects might be displayed.

[1331] Step 9: Generating the ingredient list

[1332] The server creates a list of necessary ingredients based on the generated menu.

[1333] The server sends a list of ingredients to the user's device. For example, an ingredient list such as "chicken, curry powder, various vegetables, and rice" will be displayed.

[1334] Step 10: Choosing a delivery service

[1335] The user selects a grocery delivery service.

[1336] The terminal notifies the server of its selection.

[1337] Step 11: Processing Orders and Deliveries

[1338] The server sends an order request, including a list of necessary ingredients and the delivery address, to the API of a partnered delivery service.

[1339] json

[1340] {

[1341] "address": "User's address",

[1342] "items": ["Chicken", "Curry powder", "Vegetables", "Rice"]

[1343] }

[1344] A delivery service receives the request and delivers the ingredients to the user's address.

[1345] This series of steps allows users to obtain economical and healthy meal plans and necessary ingredients without requiring specialized knowledge or time. Furthermore, meal suggestions that take into account the user's emotional state can comprehensively support their physical and mental health.

[1346] (Example 2)

[1347] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1348] Conventional menu generation systems only considered the user's dietary conditions and sale information, but did not take into account the user's emotional state. Therefore, they could not suggest menus that suited the user's mental state and could not fully meet their needs. Furthermore, few systems integrated grocery delivery services for users who could not go shopping. As a result, users were not receiving significant benefits.

[1349] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1350] In this invention, the server includes means for acquiring the latest sale information, means for acquiring allergy information and dietary conditions such as calorie restrictions entered by the user, means for recognizing the user's emotional state, means for using a generative AI model that generates a menu based on the sale information, the dietary conditions, and the emotional state, means for creating a list of necessary ingredients based on the generated menu, and means for placing an order based on the ingredient list and having the ingredients delivered. This enables the integration of menu suggestions tailored to the user's mental state and an ingredient delivery service.

[1351] 1. "Latest special offer information" refers to the latest sales information, such as prices and special offers, obtained from our partner retailers.

[1352] 2. "Allergy information" refers to information about a user's allergic reactions to specific foods.

[1353] 3. "Calorie restriction" refers to information about the limit on the total amount of calories a user can consume in a day.

[1354] 4. "Emotional state" refers to the psychological state determined by analyzing the user's facial recognition data and voice data.

[1355] 5. A "generative AI model" is an artificial intelligence model that generates a desired output when specific conditions are input.

[1356] 6. A "menu" is a specific meal plan generated based on the user's eating conditions and emotional state.

[1357] 7. A "food ingredient list" is a list containing the types and quantities of ingredients needed for the generated menu.

[1358] 8. A "delivery service" is a service in which a user orders selected ingredients and has them delivered to a specified address.

[1359] 9. A "terminal" is an electronic device (e.g., a smartphone, tablet, or personal computer) used by a user to input information or check results.

[1360] 10. A "server" is a central computer that manages and processes data for the entire system and provides various services.

[1361] This invention is a system that generates economical and healthy menus for users who cook at home, taking into account supermarket sale information and the user's dietary conditions (allergy information and calorie restrictions). Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to suggest menus that are tailored to the user's mental state. In addition, it features integration with a delivery service, enabling the delivery of ingredients to people who have difficulty going shopping, such as those raising children or the elderly.

[1362] This system includes the following main components: servers, terminals, and the hardware and software used by users.

[1363] Hardware and software to be used

[1364] 1. Server:

[1365] The server retrieves special offer information from APIs of multiple retailers and stores it in a database. It also generates meal plans using a generative AI model based on the user's dietary conditions and emotional state.

[1366] Software used: API clients, database management systems, generative AI models, etc.

[1367] 2. Terminal:

[1368] The terminal sends user data input to a server and displays the generated menu and ingredient list. It also uses a camera and microphone for emotion recognition.

[1369] Hardware to use: Smartphones, tablets, and personal computers.

[1370] Software used: emotion recognition engine, web browser or application, etc.

[1371] 3. User:

[1372] Users input allergy information and calorie restrictions via their device, and their emotional state is recognized. They also review suggested menus and ingredient lists, and select a delivery service as needed.

[1373] Specific examples of program processing

[1374] 1. Emotion recognition and acquisition of special offer information:

[1375] The user uses their device's camera and microphone to perform emotion recognition. The emotion engine determines the user's emotions in real time and recognizes them as, for example, "stress."

[1376] The server retrieves the latest sale information from partner retailers, and items such as "chicken," "curry powder," and "vegetables" are listed as sale items.

[1377] 2. Entering and saving user information:

[1378] Users enter information such as "I have a nut allergy" or "I have a 1500 calorie limit per day" through the application.

[1379] The device collects this information and sends it to the server.

[1380] The server saves the received information to the database.

[1381] 3. Menu generation:

[1382] The server generates prompt messages for the generative AI based on sale information, the user's dietary conditions, and the results of the emotion engine. Example of a prompt message: "Suggest a menu that the user is stressed. Sale information includes chicken, vegetables, and bread, and the user has a nut allergy."

[1383] The server inputs prompt text into a generative AI model, and a menu is generated, for example, "relaxing chicken soup, salad, and rice."

[1384] 4. Menu provision and adjustment:

[1385] The server sends the generated menu to the user's terminal.

[1386] The user reviews the suggested menu on their device and makes adjustments as needed.

[1387] 5. Generating the ingredient list:

[1388] The server creates a list of necessary ingredients based on the generated menu.

[1389] The server sends the ingredient list to the user's terminal.

[1390] 6. Use of delivery services:

[1391] The user selects a grocery delivery service through the app.

[1392] The device notifies the server of that information.

[1393] The server sends a list of necessary ingredients and the delivery address to the API of the partnered delivery service, and then processes the delivery.

[1394] Through the processing steps described above, this system comprehensively considers the user's current emotional state, health condition, and dietary requirements, and provides specific and practical menus and ingredients. As a result, the user can lead a healthy lifestyle both physically and mentally.

[1395] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1396] Step 1: Emotion Recognition

[1397] The user uses the device's camera and microphone to allow the system to recognize their emotional state. The device acquires facial recognition data and voice data in real time. This input data includes the user's facial expressions and voice tone.

[1398] The device sends this data to the emotion engine, which then performs analysis. For example, by analyzing the user's facial muscle movements and voice pitch, the emotion engine determines the user's emotional state as "stressed," "happy," or "tired."

[1399] Step 2: Gathering information on special offers

[1400] The server sends requests to the APIs of partner retailers to retrieve the latest sale information. This input data consists of the API endpoints of each retailer.

[1401] The server receives special offer information (e.g., "Chicken: 200 yen / 100g", "Curry powder: 100 yen / bag", "Vegetables: 150 yen / bag") in JSON format, analyzes it, and then stores the analysis results in the database.

[1402] Step 3: Enter and save user information

[1403] Users input their allergy information and dietary restrictions, such as calorie limits, through a website or application. Specific input information includes "I have a nut allergy" and "I have a 1500 calorie limit per day."

[1404] The device collects this information and sends it to the server.

[1405] The server saves the received user information to the database. This data processing stores the user's individual conditions in the database.

[1406] Step 4: Menu Generation

[1407] The server retrieves special offer information, user dietary conditions, and emotion engine results from the database. Based on this information, the generative AI generates prompt messages. The input data includes special offer information, allergy information, calorie restrictions, and emotional state.

[1408] Example of a prompt message: "When the user is feeling stressed, suggest a recommended meal. Special offers include chicken, vegetables, and bread, and the user has a nut allergy."

[1409] The server inputs a prompt message into the generative AI model and receives the generated menu. For example, a menu such as "relaxing chicken soup, salad, and rice" might be output.

[1410] Step 5: Menu provision and adjustment

[1411] The server sends the generated menu to the user's terminal. The output data is the generated menu.

[1412] The user reviews the suggested menu on their device and makes adjustments as needed. For example, the user reviews a menu of "chicken soup, salad, and rice" and its explanation of its relaxing effects.

[1413] Step 6: Generating the ingredient list

[1414] The server creates a list of necessary ingredients based on the generated menu. The input data is the generated menu.

[1415] The server sends a list of ingredients it has created to the user's terminal. The ingredient list includes "chicken, curry powder, various vegetables, and rice."

[1416] Step 7: Use a delivery service

[1417] Users select grocery delivery through the application.

[1418] The device notifies the server of that information.

[1419] The server sends the necessary information (food list and delivery address) to the API of the partnered delivery service, and the delivery process is initiated. Specifically, the server sends the food list to the API, and the package is delivered to the user's address.

[1420] In this way, the system comprehensively considers the user's emotional state, dietary conditions, and special offer information to provide the optimal menu, and further supports the easy acquisition of ingredients through home delivery services.

[1421] (Application Example 2)

[1422] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1423] Existing home cooking support systems offered menu suggestions that took into account users' allergy information and dietary restrictions such as calorie limits, but they lacked the functionality to consider the user's emotional state and suggest menus that responded to it in real time. Furthermore, there is a need to make shopping at physical stores more comfortable and effective by efficiently utilizing sale items and suggesting economical and healthy menus.

[1424] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring the latest sale information, means for acquiring allergy information and dietary conditions such as calorie restrictions entered by the user, emotion recognition means for recognizing the user's emotions, means for generating a menu based on the sale information, the dietary conditions, and the emotion recognition results, means for creating a list of necessary ingredients based on the generated menu, means for placing an order based on the ingredient list and having the ingredients delivered, and means for displaying the generated menu on the user's terminal. This enables menu suggestions that take into account the user's current emotional state and efficient shopping at physical stores.

[1425] "Latest special offer information" refers to special price information for currently available products, obtained from multiple partner retailers.

[1426] "User-entered allergy information" refers to information about a user's allergies that they provide to the system.

[1427] "Calorie restriction" refers to the upper limit of daily calorie intake set by the user.

[1428] "Dietary conditions" refer to specific conditions or restrictions related to diet, such as the user's allergy information or calorie restrictions.

[1429] "Emotion recognition technology" refers to technology that analyzes and recognizes a user's emotions in real time.

[1430] The "means for generating menus" refer to a function that suggests appropriate meal menus based on acquired special offer information, meal conditions, and emotion recognition results.

[1431] The "list of necessary ingredients" is a list of all the ingredients needed to actually prepare the generated menu.

[1432] "A means of placing an order and having ingredients delivered" refers to a function that allows users to purchase ingredients they specify and arrange for them to be delivered to a designated address.

[1433] "User's device" refers to a smartphone, smart glasses, or any other electronic device used by the user.

[1434] A "generative AI model" is an artificial intelligence model that automatically generates menus tailored to user needs based on diverse data.

[1435] To implement this invention, a server, user terminals (smartphones and smart glasses), and API connections from multiple retailers are utilized. The specific hardware and software, data processing, and data calculations required for this are described below.

[1436] Hardware and software

[1437] Server: A computer server installed in a cloud environment.

[1438] User devices: Smartphones, smart glasses (e.g., Google Glass)

[1439] Emotion recognition engine: Microsoft Azure Emotion API

[1440] Database management system: MySQL

[1441] Special Sale Information Acquisition API: APIs for various supermarkets

[1442] Generative AI: OpenAI GPT-3

[1443] Data processing and data calculation

[1444] Emotion recognition and acquisition of sale information

[1445] The device's camera and microphone are used to send user facial recognition data and voice data to an emotion recognition engine. The emotion recognition engine analyzes the user's emotions and displays the results on the device in real time. The server retrieves the latest special offer information from the APIs of partner retailers, receives it in JSON format, and stores it in a database.

[1446] Entering and saving user information

[1447] Upon initial login, users enter allergy information and calorie restrictions via their device. This information is sent to the server and stored in the database.

[1448] Menu generation

[1449] The server retrieves special offer information, user health information, and emotion recognition results from the database. Then, it inputs prompts like the following into the generative AI to generate a menu.

[1450] Example: Prompt text to input to a generative AI

[1451] The user has a dairy allergy and a daily calorie restriction of 1800 calories. The user's current emotional state is fatigue. Please suggest a meal consisting of stir-fried chicken and broccoli, a salad, and whole-wheat bread, all of which are known to help with fatigue recovery.

[1452] The generated menu is saved to a database by the server.

[1453] Menu provision and adjustment

[1454] Users can view the generated menu by wearing smart glasses or using a smartphone app in the store. They can adjust the menu as needed.

[1455] Generating an ingredient list

[1456] Based on the proposed menu, the server extracts the necessary ingredients from the database and displays them on the user's terminal.

[1457] Use of delivery service

[1458] When a user selects a delivery service, the server retrieves the necessary information from the database and sends it to the API of the partnered delivery service. The delivery service then delivers the specified ingredients to the user's address.

[1459] As a concrete example, if a user wears smart glasses and the emotion recognition engine detects "fatigue," the server retrieves "chicken," "broccoli," and "olive oil" as sale items from "Supermarket A." Based on the information the user has entered, such as "dairy allergy" and "1800 calorie limit per day," the generative AI generates a menu of "stir-fried chicken and broccoli, salad, and whole wheat bread." The user confirms this menu, and "chicken, broccoli, olive oil, and whole wheat bread" are displayed as necessary ingredients. If the user selects a delivery service, the ingredients are delivered to their home.

[1460] As a result, users can receive menu suggestions that take their emotional state into consideration and achieve efficient shopping at physical stores.

[1461] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1462] Step 1:

[1463] Users log in to the system via smart glasses or smartphones. After logging in, they enter initial settings such as allergy information and dietary conditions including calorie restrictions. The entered information is sent from the device to the server and stored in the database.

[1464] Input: User allergy information, calorie restriction settings

[1465] Data processing: Information collection and format conversion

[1466] Output: User information stored in the database

[1467] Step 2:

[1468] The user uses the smart glasses' camera and microphone to send facial recognition data and voice data to the emotion recognition engine. The emotion recognition engine analyzes this data and recognizes the user's emotional state in real time. The recognition results are displayed on the device.

[1469] Input: User's facial recognition data, voice data

[1470] Data processing: Data analysis using an emotion recognition engine.

[1471] Output: Real-time emotional state

[1472] Step 3:

[1473] The server retrieves the latest sale information from the APIs of each partner retailer. The retrieved sale information is received in JSON format, the server analyzes the data, and saves the necessary information to the database.

[1474] Input: Special offer information from retailer API

[1475] Data processing: Parsing JSON data and storing it in a database.

[1476] Output: Special sale information stored in the database

[1477] Step 4:

[1478] The server retrieves the user's dietary conditions (allergy information, calorie restrictions), emotion recognition results, and special offer information from the database. Based on this information, it generates prompt messages, which are then input into a generative AI to generate a menu.

[1479] Input: User's dietary conditions, emotional state, special offer information

[1480] Data processing: Prompt processing by generative AI

[1481] Output: Generated menu

[1482] Examples of prompt statements:

[1483] The user has a dairy allergy and a daily calorie restriction of 1800 calories. The user's current emotional state is fatigue. Please suggest a meal consisting of stir-fried chicken and broccoli, a salad, and whole-wheat bread, all of which are known to help with fatigue recovery.

[1484] Step 5:

[1485] The server sends the generated menu to the user's terminal. The user can review the menu and make adjustments as needed. The adjustment results are then sent back to the server.

[1486] Input: Generated menu

[1487] Data processing: Information transmission and user adjustments.

[1488] Output: Adjusted final menu

[1489] Step 6:

[1490] The server creates a list of necessary ingredients based on the final menu and sends it to the terminal. The user reviews the list and selects the ingredients they want to order.

[1491] Input: Final Menu

[1492] Data processing: Extraction and creation of a list of necessary ingredients.

[1493] Output: Ingredient list

[1494] Step 7:

[1495] When a user selects grocery delivery, the device notifies the server of this information. The server then sends the necessary information to the API of its partner delivery service and initiates the process of delivering the groceries to the user's address.

[1496] Input: Ingredient list, delivery selection information

[1497] Data processing: Sending data to delivery service APIs

[1498] Output: Food items delivered to the user's address

[1499] The above outlines the specific processing steps of the program of this invention. Through these steps, users can achieve healthier and more efficient shopping based on personalized suggestions that take their emotional state into consideration.

[1500] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1501] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1502] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1503] [Fourth Embodiment]

[1504] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1505] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1506] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1507] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1508] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1509] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1510] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1511] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1512] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1513] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1514] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1515] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1516] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1517] This invention is a system that targets people who cook at home and generates economical and healthy menus by taking into account supermarket sale information and dietary conditions specified by the user (allergy information and calorie restrictions). Furthermore, by linking with delivery services, it is possible to deliver ingredients to people who have difficulty going shopping, such as those raising children or the elderly.

[1518] This system mainly consists of the following elements:

[1519] 1. Gathering information on special offers

[1520] 2. Entering and saving user information

[1521] 3. Menu generation

[1522] 4. Providing a list of ingredients

[1523] 5. Use of delivery services

[1524] Program processing flow

[1525] Gathering information on special offers

[1526] The server uses a script that runs at specific times to retrieve special offer information from the APIs of multiple partner retailers. For example, the server sends a GET request to a specific URL at 3 AM every day and saves the received JSON data to the database. In this way, the latest special offer information is always stored in the database.

[1527] Entering user information

[1528] Users input dietary information such as allergy details and calorie restrictions through applications or websites. The device receives this information and sends it to the server. For example, a user might enter information such as "I have a nut allergy" and "I have a 1500 calorie limit per day" into an input form. The device then sends this data to the server as a POST request in JSON format.

[1529] User information storage

[1530] The server stores the received user information in a database. This makes it easy to reuse information that the user may need in the future.

[1531] Menu generation

[1532] The server uses a generative AI to generate menus based on user information and acquired sale information. For example, the server sends the aforementioned user information and sale information to the AI, which then suggests a menu like "chicken curry, salad, and rice."

[1533] Menu

[1534] The generated menu is sent from the server to the user's device and displayed visually on the device. Users can check the menu and its details on devices such as PCs and smartphones.

[1535] Providing a list of ingredients

[1536] Based on the menu, the server creates a list of necessary ingredients and sends it to the user's terminal. For example, an ingredient list such as "chicken, curry powder, various vegetables, rice" will be displayed.

[1537] Use of delivery service

[1538] When a user requests delivery, they make a selection through their device, and that information is sent to the server. The server then sends the necessary information to the partner delivery service, and the groceries are delivered to the specified address. For example, the server might send the "delivery address and list of required groceries" as a POST request to the delivery company's API.

[1539] Specific example

[1540] In reality, the following scenario is possible:

[1541] 1. Gathering information on special offers:

[1542] The server retrieves special offer information from "Store A" and "Store B," and items such as "chicken," "curry powder," and "vegetables" are listed as special offer products.

[1543] 2. Entering and saving user information:

[1544] The user accesses the website and enters "nut allergy" as allergy information and "1500 calories" as calorie restriction.

[1545] 3. Menu generation:

[1546] Based on the aforementioned special offer information and user information, the server uses a generative AI to generate a menu consisting of "chicken curry, salad, and rice."

[1547] 4. Menu provision:

[1548] The server sends this menu to the user's device, and the user checks the menu on their smartphone.

[1549] 5. Provide a list of ingredients:

[1550] Based on the menu, a list of ingredients such as "chicken, curry powder, various vegetables, and rice" will be displayed on the user's device.

[1551] 6. Use of delivery services:

[1552] The user selects a grocery delivery service, and the server initiates the process of delivering the groceries to the specified address.

[1553] This system allows users to plan economical and healthy meals without hassle and have ingredients delivered to their homes. This helps reduce food waste and loss, and also allows for better health management.

[1554] The following describes the processing flow.

[1555] Step 1: Gathering sale information

[1556] The server uses a script that runs periodically to retrieve special offer information from the APIs of partner retailers.

[1557] The server sends GET requests to each retailer's API endpoint and parses the JSON data received as a response.

[1558] The server analyzes the sale information and saves it to the database. This ensures that the latest sale information is always available.

[1559] Step 2: Enter user information

[1560] The user logs in to the application or website.

[1561] The device displays an input form to the user and collects allergy information, calorie restrictions, and other dietary conditions.

[1562] The user enters this information and clicks the submit button.

[1563] Step 3: Submit User Information

[1564] The device collects user information and sends it to the server as a POST request in JSON format.

[1565] For example, the device sends the following data to the server.

[1566] json

[1567] {

[1568] "user_id": "12345",

[1569] "allergies": ["nuts"],

[1570] "calorie_limit": 1500

[1571] }

[1572] Step 4: Saving User Information

[1573] The server stores the received user information in the database.

[1574] The server associates allergy information and calorie restrictions based on the user ID.

[1575] Step 5: Submit a menu generation request

[1576] The server sends a request to the generative AI based on the user's dietary requirements and the latest special offers.

[1577] Request data includes allergy information, calorie restrictions, and a list of sale items.

[1578] json

[1579] {

[1580] "allergies": ["nuts"],

[1581] "calorie_limit": 1500,

[1582] "Specials": ["Chicken", "Curry powder", "Vegetables"]

[1583] }

[1584] Step 6: Menu generation using generative AI

[1585] The generative AI analyzes the received request data and generates the optimal menu.

[1586] As an example, generate a menu consisting of "chicken curry, salad, and rice."

[1587] Step 7: Serving the menu

[1588] The server receives the generated menu data and sends it to the user's terminal.

[1589] The device displays the generated menu to the user. The user can view the menu details and cooking instructions.

[1590] Step 8: Generating the ingredient list

[1591] The server creates a list of necessary ingredients based on the generated menu.

[1592] The server sends the ingredient list to the user's terminal.

[1593] json

[1594] {

[1595] "items": ["Chicken", "Curry powder", "Vegetables", "Rice"]

[1596] }

[1597] Step 9: Choosing a delivery service

[1598] The user selects a grocery delivery service on their device.

[1599] The terminal notifies the server of the user's selection.

[1600] Step 10: Processing Orders and Deliveries

[1601] The server sends an order request, including a list of necessary ingredients and the delivery address, to the API of a partnered delivery service.

[1602] json

[1603] {

[1604] "address": "User's address",

[1605] "items": ["Chicken", "Curry powder", "Vegetables", "Rice"]

[1606] }

[1607] A delivery service receives the request and delivers the ingredients to the user's address.

[1608] This series of steps allows users to automatically obtain economical and healthy meal plans and necessary ingredients, which are then delivered to their homes. This eliminates the hassle of shopping and reduces food waste and unnecessary expenses.

[1609] (Example 1)

[1610] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1611] With existing systems, users have to manually collect sale information and create meal plans while considering allergy information and calorie restrictions, which is time-consuming. This is even more difficult for people who have difficulty going grocery shopping, such as the elderly and those with young children. This can lead to wasted food, food loss, and difficulties in managing one's health.

[1612] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1613] In this invention, the server includes means for collecting the latest sale information, means for obtaining allergy information and dietary conditions such as calorie restrictions entered by the user, means for generating a menu using a generation AI model based on the sale information and dietary conditions, means for creating a list of necessary ingredients based on the generated menu, and means for placing an order based on the ingredient list and having the ingredients delivered. This makes it possible for users to obtain economical and healthy menus without any hassle and have the ingredients delivered to their homes.

[1614] "Special sale information" refers to information about discounts and special offers on products that can be obtained from partner retailers.

[1615] "User information" refers to information about dietary conditions such as allergy information and calorie restrictions that users enter.

[1616] A "generative AI model" is an artificial intelligence model used to generate menus based on special offer information and user information.

[1617] A "menu" refers to a combination of meals suggested by an AI model based on special offer information and user data.

[1618] A "food ingredient list" is a list of ingredients needed based on the generated menu.

[1619] "Delivery service" refers to the means of arranging for the delivery of groceries ordered by a user to a specified address.

[1620] A "terminal" is an electronic device used by users to input information or to display generated menus and ingredient lists.

[1621] The "server" is a central computer system that performs tasks such as collecting special offer information, storing user information, operating the generated AI model, creating menus and ingredient lists, and arranging delivery methods.

[1622] A "database" is a system for storing and managing data such as special sale information and user information.

[1623] An "API" is an interface that allows different software components to interact with each other.

[1624] This invention is a system that automates everything from generating menus to purchasing and delivering ingredients, which are necessary when a user plans a meal. Specific embodiments of this system are shown below.

[1625] Gathering information on special offers

[1626] The server uses a script that automatically runs at a specific time (for example, 3 AM every day) to retrieve special offer information from the APIs of multiple partner retailers. This ensures that the latest special offer information is always stored in the database. The server uses the Python requests library to send GET requests to each retailer's API endpoint and saves the returned JSON data to a database such as MySQL.

[1627] Entering user information

[1628] Users enter their allergy information and dietary conditions such as calorie restrictions through the application or website. This information is received by the device and sent to the server as a POST request in JSON format. When the user enters the necessary information in a form on the browser and clicks the "Submit" button, the front-end JavaScript code converts the data into JSON format and sends it to the server using the axios library.

[1629] User information storage

[1630] The server parses the user information it receives and stores it in a database. This makes it easy to reuse information that the user might need in the future. For example, the server parses JSON data received via a Flask or Django endpoint and saves it to a database such as MySQL.

[1631] Menu generation

[1632] The server generates a menu using a generative AI model (e.g., GPT-4) based on special offer information and user information. The server sends a prompt to the OpenAI API endpoint, and the AI ​​generates the menu based on that prompt. Examples of prompts include "Special offer information: chicken, curry powder, vegetables" and "User information: nut allergy, 1500 calorie limit." This generates a menu such as "chicken curry, salad, rice."

[1633] Menu

[1634] The generated menu is sent from the server to the user's device and displayed visually on the device. Users can view the menu and its details on devices such as PCs and smartphones. The server returns JSON data as a Flask or Django response, and the frontend displays the received data in HTML format.

[1635] Providing a list of ingredients

[1636] The server creates a list of necessary ingredients based on the generated menu and sends it to the user's terminal. An ingredient list such as "chicken, curry powder, various vegetables, rice" is displayed. The server selects the names of the necessary ingredients from the database to generate a list and sends it to the terminal in JSON format. The frontend receives the list data and displays it so that the user can visually confirm it.

[1637] Use of delivery service

[1638] When a user requests delivery, they make a selection through their device, and that information is sent to the server. The server sends the necessary information to the partner delivery service, and the groceries are delivered to the specified address. The server sends the "delivery address and list of required groceries" as a POST request in JSON format to the delivery company's API endpoint, and the actual delivery process is carried out.

[1639] This system allows users to plan economical and healthy meals without hassle and have ingredients delivered to their homes. This automation can reduce food waste and spoilage, and even improve health management.

[1640] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1641] Step 1: Gathering sale information

[1642] Processing flow:

[1643] The server automatically executes a script at a specific time (for example, 3 AM every day) to send GET requests to the APIs of multiple partner retailers. For example, the server accesses "https: / / api.storeA.com / sales" and "https: / / api.storeB.com / sales".

[1644] Specific actions:

[1645] The server sends a GET request using the Python requests library and receives JSON data returned as a response.

[1646] input:

[1647] The API endpoint URL for a specific retail store.

[1648] output:

[1649] Special sale information data in JSON format.

[1650] Data processing:

[1651] The server parses the received JSON data and executes an INSERT query to save it to the MySQL database.

[1652] Step 2: Enter user information

[1653] Processing flow:

[1654] Users enter dietary conditions such as allergy information and calorie restrictions through the application or website. For example, they might enter "I have a nut allergy" or "I need to limit myself to 1500 calories per day."

[1655] Specific actions:

[1656] The user enters information into a form in their browser and clicks the "Submit" button. JavaScript code on the frontend converts the input data into JSON format and sends a POST request to the server using the axios library.

[1657] input:

[1658] Allergy information and dietary conditions such as calorie restrictions entered by the user.

[1659] output:

[1660] User information data in JSON format.

[1661] Data processing:

[1662] The terminal parses the entered information in JSON format and sends it to the server.

[1663] Step 3: Saving User Information

[1664] Processing flow:

[1665] The server parses the received user information and stores it in the database. For example, it stores data corresponding to the fields "user_id", "allergies", and "calorie_limit".

[1666] Specific actions:

[1667] The server parses the JSON data received via the Flask or Django endpoint and saves it to the MySQL database using INSERT queries.

[1668] input:

[1669] User information data in JSON format.

[1670] output:

[1671] User information stored in the database.

[1672] Data processing:

[1673] The server parses the JSON data and inserts it into the corresponding database fields.

[1674] Step 4: Menu Generation

[1675] Processing flow:

[1676] The server sends prompts to a generating AI model (e.g., GPT-4) based on special offer information and user information, and the AI ​​generates a menu. For example, it might send prompts such as "Special offer information: Chicken, curry powder, vegetables" and "User information: Nut allergy, 1500 calorie limit".

[1677] Specific actions:

[1678] The server sends prompts to the OpenAI API endpoint using Python code and receives responses from the AI.

[1679] input:

[1680] A prompt message containing special offer information and user information.

[1681] output:

[1682] The generated menu data.

[1683] Data processing:

[1684] The server generates a prompt and sends it to the AI ​​model. The AI's response is analyzed and saved in JSON format.

[1685] Step 5: Serving the menu

[1686] Processing flow:

[1687] The generated menu is sent from the server to the user's device and displayed visually on the device. Users can check the details of the menu on devices such as PCs and smartphones.

[1688] Specific actions:

[1689] The server returns JSON data as a response from Flask or Django, which is then parsed on the frontend and displayed in HTML format.

[1690] input:

[1691] The generated menu data.

[1692] output:

[1693] Menu information displayed on the user's terminal.

[1694] Data processing:

[1695] The device parses the JSON data and generates HTML for visual display.

[1696] Step 6: Provide the ingredient list

[1697] Processing flow:

[1698] The server creates a list of necessary ingredients based on the generated menu and sends it to the user's terminal. For example, it might generate a list such as "chicken, curry powder, various vegetables, rice."

[1699] Specific actions:

[1700] The server selects the necessary ingredient information from the database, generates a list, and sends it to the terminal in JSON format.

[1701] input:

[1702] The generated menu data.

[1703] output:

[1704] A list of ingredients displayed on the user's terminal.

[1705] Data processing:

[1706] The server retrieves ingredient information from the database, creates a list, and sends it.

[1707] Step 7: Use a delivery service

[1708] Processing flow:

[1709] When a user requests grocery delivery, they send their selection information to a server via their device. The server then transmits the necessary information to a partner delivery service, and the groceries are delivered to the specified address.

[1710] Specific actions:

[1711] The user clicks the "Request Delivery" button in the application's UI, and the frontend sends a POST request to the server. The server then sends the delivery information to the delivery company's API endpoint via a POST request.

[1712] input:

[1713] User delivery preference information.

[1714] output:

[1715] Delivery information sent to the courier company.

[1716] Data processing:

[1717] The server analyzes the user's delivery information and sends it to the delivery company's API in the appropriate format.

[1718] (Application Example 1)

[1719] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1720] Traditional systems required users to manually search for sale information and create meal plans considering allergy information and calorie restrictions. Furthermore, it was difficult to effectively utilize sale information to generate economical meal plans, and the procedures for using food delivery services were cumbersome. Therefore, there was a need to simultaneously achieve effective use of sale information and health management.

[1721] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1722] In this invention, the server includes means for acquiring the latest special offer information, means for acquiring allergy information and dietary conditions such as calorie restrictions entered by the user, means for utilizing a generative AI model that generates menus based on the special offer information and the dietary conditions, means for creating a list of necessary ingredients based on the generated menus, and means for placing orders and having the ingredients delivered based on the ingredient list. As a result, users can effectively utilize special offer information, automatically generate menus suited to their individual dietary conditions, and easily purchase and have the necessary ingredients delivered.

[1723] "Means of obtaining the latest sale information" refers to a system or method that can regularly collect and store sale information provided by supermarkets and other retailers.

[1724] "Means for obtaining user-entered allergy information and dietary conditions such as calorie restrictions" refers to an interface for users to input individual dietary restrictions, such as allergies to specific foods or calorie restrictions, and a system for processing that information.

[1725] "Means of using a generative AI model that generates menus based on the aforementioned special offer information and the aforementioned meal conditions" refers to a system or method that uses an artificial intelligence model that automatically generates appropriate menus based on collected special offer information and the user's meal conditions as input.

[1726] "Means for creating a list of necessary ingredients based on the generated menu" refers to a system or method that has the function of compiling a list of necessary ingredients based on the generated menu.

[1727] "Means of ordering and delivering ingredients based on the aforementioned ingredient list" refers to a system or method that allows a user to order ingredients online based on a generated ingredient list and have those ingredients delivered to a specified address.

[1728] This invention is a system that allows users to plan healthy and economical meals without hassle and efficiently obtain the necessary ingredients. The system consists of a server, a user terminal, and partner retailers and delivery services.

[1729] First, the server periodically retrieves special offer information from multiple partner retailers. This information is collected via APIs provided by the retailers and stored in the server's database. The special offer information is retrieved in JSON format, and the database always maintains the most up-to-date information.

[1730] Next, the user enters allergy information and dietary conditions such as calorie restrictions into a dedicated application via their smartphone or computer. This information is sent from the device to a server and stored in a database.

[1731] The server uses a generative AI model to generate a menu tailored to the user, based on saved sale information and the user's dietary conditions. In this case, the generative AI model uses the sale information and user information as input. For example, based on user information such as "nut allergy, 1500 calories per day" and sale items such as "chicken, curry powder, vegetables," the model suggests a menu such as "chicken curry, salad, rice."

[1732] Furthermore, the server creates a list of necessary ingredients based on the generated menu and sends this list to the user's device. This list is displayed in a format that can be visually confirmed on the user's device.

[1733] When a user wants groceries delivered, they place an order through their device. This information is sent to the server, which uses the API of a partnered delivery service to arrange for the groceries to be delivered to the specified address.

[1734] For example, a menu is generated based on the user's input conditions, such as "nut allergy, 1500 calories per day," and special offer information, and a list of corresponding ingredients is displayed on the terminal. If the user selects ingredient delivery, the server automatically places an order with the delivery service, and the ingredients are delivered to the user's address.

[1735] An example of a prompt statement is as follows:

[1736] "User information: Has a nut allergy, daily calorie intake of 1500 kcal. Special offer information: Chicken, curry powder, vegetables."

[1737] This system allows users to efficiently plan healthy meals using sale information and easily obtain the necessary ingredients. As a result, it reduces the effort required from users, facilitates the effective use of sale information, improves health management, and reduces food waste.

[1738] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1739] Step 1:

[1740] The server periodically retrieves special offer information from the APIs of partner retailers. Specifically, the server sends a GET request to a designated URL of a retailer at a fixed time each day and receives the special offer information in JSON format. This JSON data is stored in a database. The input data is the special offer information retrieved from the retailer, and the output data is the special offer information stored in the database.

[1741] Step 2:

[1742] Users input their allergy information and dietary conditions, such as calorie restrictions, through a smartphone or computer application. This information is sent from the device to the server and stored in a database in JSON format. The input data is the user's dietary conditions, and the output data is the user's dietary conditions stored in the database.

[1743] Step 3:

[1744] The server sends special offer information retrieved from the database and the user's dietary conditions as input to the generating AI model. At this time, it generates a prompt message and makes a request to the AI ​​model. For example, it sends data in the format: "User information: Nut allergy, 1500 calories per day. Special offer information: Chicken, curry powder, vegetables." The generating AI model proposes an appropriate menu, which the server receives. The input data consists of the user's dietary conditions and special offer information, while the output data is the generated menu.

[1745] Step 4:

[1746] The server creates a list of necessary ingredients based on the generated menu. This list is generated in JSON format and sent to the user's terminal. Specifically, it analyzes the menu data obtained from the generation AI model and lists the necessary ingredients. The input data is the generated menu, and the output data is the ingredient list.

[1747] Step 5:

[1748] The user reviews the displayed list of ingredients and selects their delivery preferences through the application. This information is sent from the terminal to the server. The input data is the user's delivery preference information, and the output data is the delivery preference information stored on the server.

[1749] Step 6:

[1750] The server places an order with a partner delivery service via its API, based on the user's delivery preferences and ingredient list. Specifically, it converts the delivery address and required ingredient list into an API request format and sends it as a POST request. The delivery service then returns confirmation that the order has been received. The input data consists of the delivery preferences and ingredient list, while the output data is the order confirmation sent to the delivery service.

[1751] Through these steps, users can easily take advantage of special offers, create healthy and economical meal plans, and efficiently obtain the necessary ingredients.

[1752] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1753] This invention is a system that targets people who cook at home and generates economical and healthy menus that take into account supermarket sale information and the user's dietary conditions (allergy information and calorie restrictions). Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to suggest menus that are tailored to the user's mental state. In addition, it features integration with a delivery service, making it possible to deliver ingredients to people who have difficulty going shopping, such as those raising children or the elderly.

[1754] Program processing flow

[1755] Emotion recognition by an emotion engine

[1756] This system includes an emotion engine that recognizes the user's emotions. The emotion engine analyzes facial recognition data and voice data acquired from the user's device to determine the user's emotional state in real time.

[1757] Gathering information on special offers

[1758] The server retrieves special offer information from APIs of multiple partner retailers. This information is updated daily and stored in a database. The server sends requests to each retailer's API endpoint, parses the JSON data received as a response, and stores it in the database.

[1759] Entering and saving user information

[1760] Users enter allergy information, calorie restrictions, current emotional state, etc., through a website or application. The device collects this information and sends it to the server. The server stores the received information in a database.

[1761] Menu generation

[1762] Based on the special offer information acquired by the server, the user's dietary preferences, and the user's emotional state recognized by the emotion engine, a generative AI is used to generate a menu. For example, if the user is feeling stressed, a menu using ingredients with relaxing effects will be suggested.

[1763] Menu provision and adjustment

[1764] The generated menu is sent from the server to the user's terminal and displayed on the terminal. Users can check menus that accommodate allergies and calorie restrictions. Menus based on the user's emotional state are also suggested.

[1765] Generating an ingredient list

[1766] The server creates a list of necessary ingredients based on the generated menu and sends it to the user's terminal. The user can then review the list and proceed with placing an order.

[1767] Use of delivery service

[1768] When a user selects a delivery service, the device notifies the server of that information. The server then sends a list of necessary ingredients and the delivery address to the API of the partnered delivery service, and initiates the process of delivering the ingredients to the user's address.

[1769] Specific example

[1770] 1. Emotion recognition and acquisition of special offer information:

[1771] The user uses the device's camera and microphone to recognize their emotions. The emotion engine analyzes the user's facial recognition data and voice data and recognizes it as "stress."

[1772] The server retrieves the latest sale information from "Store A" and "Store B," and items such as "chicken," "curry powder," and "vegetables" are listed as sale items.

[1773] 2. Entering and saving user information:

[1774] The user enters "I have a nut allergy" and "I'm limiting myself to 1500 calories a day" into the application.

[1775] The server saves this information to the database.

[1776] 3. Menu generation:

[1777] Based on sale information, the user's meal preferences, and the results of the emotion engine, the server uses generative AI to generate a menu of "relaxing chicken soup, salad, and rice."

[1778] 4. Menu provision and adjustment:

[1779] The server sends the generated menu to the user's terminal, and the user checks the menu ("chicken soup, salad, rice") and the explanation of its relaxing effects in the application.

[1780] 5. Generating the ingredient list:

[1781] Based on the menu, a list of ingredients such as "chicken, curry powder, various vegetables, and rice" is displayed on the user's device.

[1782] 6. Use of delivery services:

[1783] The user selects a grocery delivery service, and the server sends the necessary information to the partnered delivery service.

[1784] A delivery service delivers groceries to the user's address.

[1785] As described above, this system comprehensively considers the user's current emotional state, health condition, and dietary conditions, and provides specific and practical menus and ingredients. As a result, users can lead a healthy lifestyle both physically and mentally.

[1786] The following describes the processing flow.

[1787] Step 1: Gathering sale information

[1788] The server uses a script that runs periodically to retrieve special offer information from the APIs of multiple partner retailers. Specifically, the server sends GET requests to each API endpoint at 3 AM every day and parses the JSON data received as a response.

[1789] The server saves the received sale information to a database. This ensures that the database always contains the most up-to-date sale information.

[1790] Step 2: Enter user information

[1791] The user logs in to the application or website.

[1792] The device displays an input form to the user, collecting allergy information, calorie restrictions, and other dietary conditions. For example, the user might enter information such as "I have a nut allergy" or "I have a 1500 calorie limit per day."

[1793] Review the information collected by the user and click the submit button.

[1794] Step 3: Submit User Information

[1795] The device collects user information and sends it to the server as a POST request in JSON format.

[1796] For example, the device sends the following data to the server.

[1797] json

[1798] {

[1799] "user_id": "12345",

[1800] "allergies": ["nuts"],

[1801] "calorie_limit": 1500

[1802] }

[1803] Step 4: Saving User Information

[1804] The server stores the received user information in a database. Allergy information and calorie restrictions are associated based on the user ID.

[1805] Step 5: Emotion recognition by the emotion engine

[1806] The device activates its function to acquire the user's facial recognition data and voice data.

[1807] The user shows their face to the device's camera and microphone and speaks.

[1808] The device sends facial recognition data and voice data to the emotion engine, which then analyzes it.

[1809] The server receives the analysis results and determines the user's current emotional state. For example, it might determine that the user is "stressed."

[1810] Step 6: Submit a menu generation request

[1811] The server sends a request to the generative AI based on the user's meal conditions, the results of the emotion engine, and special offer information.

[1812] Request data includes allergy information, calorie restrictions, a list of sale items, and the user's current emotional state.

[1813] json

[1814] {

[1815] "allergies": ["nuts"],

[1816] "calorie_limit": 1500,

[1817] "specials": ["chicken", "curry powder", "vegetables"],

[1818] "emotion": "stress"

[1819] }

[1820] Step 7: Menu generation using generative AI

[1821] The generative AI analyzes the received request data and generates a menu that is appropriate for the user's emotional state.

[1822] For example, if the emotion engine's result is "stress," it will generate a menu of "chicken soup, salad, and rice" which have a relaxing effect.

[1823] Step 8: Serving the menu

[1824] The server receives the generated menu data and sends it to the user's terminal.

[1825] The device displays the menu to the user, who can then view the details. For example, a menu such as "chicken soup, salad, and rice" and a description of its relaxing effects might be displayed.

[1826] Step 9: Generating the ingredient list

[1827] The server creates a list of necessary ingredients based on the generated menu.

[1828] The server sends a list of ingredients to the user's device. For example, an ingredient list such as "chicken, curry powder, various vegetables, and rice" will be displayed.

[1829] Step 10: Choosing a delivery service

[1830] The user selects a grocery delivery service.

[1831] The terminal notifies the server of its selection.

[1832] Step 11: Processing Orders and Deliveries

[1833] The server sends an order request, including a list of necessary ingredients and the delivery address, to the API of a partnered delivery service.

[1834] json

[1835] {

[1836] "address": "User's address",

[1837] "items": ["Chicken", "Curry powder", "Vegetables", "Rice"]

[1838] }

[1839] A delivery service receives the request and delivers the ingredients to the user's address.

[1840] This series of steps allows users to obtain economical and healthy meal plans and necessary ingredients without requiring specialized knowledge or time. Furthermore, meal suggestions that take into account the user's emotional state can comprehensively support their physical and mental health.

[1841] (Example 2)

[1842] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1843] Conventional menu generation systems only considered the user's dietary conditions and sale information, but did not take into account the user's emotional state. Therefore, they could not suggest menus that suited the user's mental state and could not fully meet their needs. Furthermore, few systems integrated grocery delivery services for users who could not go shopping. As a result, users were not receiving significant benefits.

[1844] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1845] In this invention, the server includes means for acquiring the latest sale information, means for acquiring allergy information and dietary conditions such as calorie restrictions entered by the user, means for recognizing the user's emotional state, means for using a generative AI model that generates a menu based on the sale information, the dietary conditions, and the emotional state, means for creating a list of necessary ingredients based on the generated menu, and means for placing an order based on the ingredient list and having the ingredients delivered. This enables the integration of menu suggestions tailored to the user's mental state and an ingredient delivery service.

[1846] 1. "Latest special offer information" refers to the latest sales information, such as prices and special offers, obtained from our partner retailers.

[1847] 2. "Allergy information" refers to information about a user's allergic reactions to specific foods.

[1848] 3. "Calorie restriction" refers to information about the limit on the total amount of calories a user can consume in a day.

[1849] 4. "Emotional state" refers to the psychological state determined by analyzing the user's facial recognition data and voice data.

[1850] 5. A "generative AI model" is an artificial intelligence model that generates a desired output when specific conditions are input.

[1851] 6. A "menu" is a specific meal plan generated based on the user's eating conditions and emotional state.

[1852] 7. A "food ingredient list" is a list containing the types and quantities of ingredients needed for the generated menu.

[1853] 8. A "delivery service" is a service in which a user orders selected ingredients and has them delivered to a specified address.

[1854] 9. A "terminal" is an electronic device (e.g., a smartphone, tablet, or personal computer) used by a user to input information or check results.

[1855] 10. A "server" is a central computer that manages and processes data for the entire system and provides various services.

[1856] This invention is a system that generates economical and healthy menus for users who cook at home, taking into account supermarket sale information and the user's dietary conditions (allergy information and calorie restrictions). Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to suggest menus that are tailored to the user's mental state. In addition, it features integration with a delivery service, enabling the delivery of ingredients to people who have difficulty going shopping, such as those raising children or the elderly.

[1857] This system includes the following main components: servers, terminals, and the hardware and software used by users.

[1858] Hardware and software to be used

[1859] 1. Server:

[1860] The server retrieves special offer information from APIs of multiple retailers and stores it in a database. It also generates meal plans using a generative AI model based on the user's dietary conditions and emotional state.

[1861] Software used: API clients, database management systems, generative AI models, etc.

[1862] 2. Terminal:

[1863] The terminal sends user data input to a server and displays the generated menu and ingredient list. It also uses a camera and microphone for emotion recognition.

[1864] Hardware to use: Smartphones, tablets, and personal computers.

[1865] Software used: emotion recognition engine, web browser or application, etc.

[1866] 3. User:

[1867] Users input allergy information and calorie restrictions via their device, and their emotional state is recognized. They also review suggested menus and ingredient lists, and select a delivery service as needed.

[1868] Specific examples of program processing

[1869] 1. Emotion recognition and acquisition of special offer information:

[1870] The user uses their device's camera and microphone to perform emotion recognition. The emotion engine determines the user's emotions in real time and recognizes them as, for example, "stress."

[1871] The server retrieves the latest sale information from partner retailers, and items such as "chicken," "curry powder," and "vegetables" are listed as sale items.

[1872] 2. Entering and saving user information:

[1873] Users enter information such as "I have a nut allergy" or "I have a 1500 calorie limit per day" through the application.

[1874] The device collects this information and sends it to the server.

[1875] The server saves the received information to the database.

[1876] 3. Menu generation:

[1877] The server generates prompt messages for the generative AI based on sale information, the user's dietary conditions, and the results of the emotion engine. Example of a prompt message: "Suggest a menu that the user is stressed. Sale information includes chicken, vegetables, and bread, and the user has a nut allergy."

[1878] The server inputs prompt text into a generative AI model, and a menu is generated, for example, "relaxing chicken soup, salad, and rice."

[1879] 4. Menu provision and adjustment:

[1880] The server sends the generated menu to the user's terminal.

[1881] The user reviews the suggested menu on their device and makes adjustments as needed.

[1882] 5. Generating the ingredient list:

[1883] The server creates a list of necessary ingredients based on the generated menu.

[1884] The server sends the ingredient list to the user's terminal.

[1885] 6. Use of delivery services:

[1886] The user selects a grocery delivery service through the app.

[1887] The device notifies the server of that information.

[1888] The server sends a list of necessary ingredients and the delivery address to the API of the partnered delivery service, and then processes the delivery.

[1889] Through the processing steps described above, this system comprehensively considers the user's current emotional state, health condition, and dietary requirements, and provides specific and practical menus and ingredients. As a result, the user can lead a healthy lifestyle both physically and mentally.

[1890] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1891] Step 1: Emotion Recognition

[1892] The user uses the device's camera and microphone to allow the system to recognize their emotional state. The device acquires facial recognition data and voice data in real time. This input data includes the user's facial expressions and voice tone.

[1893] The device sends this data to the emotion engine, which then performs analysis. For example, by analyzing the user's facial muscle movements and voice pitch, the emotion engine determines the user's emotional state as "stressed," "happy," or "tired."

[1894] Step 2: Gathering information on special offers

[1895] The server sends requests to the APIs of partner retailers to retrieve the latest sale information. This input data consists of the API endpoints of each retailer.

[1896] The server receives special offer information (e.g., "Chicken: 200 yen / 100g", "Curry powder: 100 yen / bag", "Vegetables: 150 yen / bag") in JSON format, analyzes it, and then stores the analysis results in the database.

[1897] Step 3: Enter and save user information

[1898] Users input their allergy information and dietary restrictions, such as calorie limits, through a website or application. Specific input information includes "I have a nut allergy" and "I have a 1500 calorie limit per day."

[1899] The device collects this information and sends it to the server.

[1900] The server saves the received user information to the database. This data processing stores the user's individual conditions in the database.

[1901] Step 4: Menu Generation

[1902] The server retrieves special offer information, user dietary conditions, and emotion engine results from the database. Based on this information, the generative AI generates prompt messages. The input data includes special offer information, allergy information, calorie restrictions, and emotional state.

[1903] Example of a prompt message: "When the user is feeling stressed, suggest a recommended meal. Special offers include chicken, vegetables, and bread, and the user has a nut allergy."

[1904] The server inputs a prompt message into the generative AI model and receives the generated menu. For example, a menu such as "relaxing chicken soup, salad, and rice" might be output.

[1905] Step 5: Menu provision and adjustment

[1906] The server sends the generated menu to the user's terminal. The output data is the generated menu.

[1907] The user reviews the suggested menu on their device and makes adjustments as needed. For example, the user reviews a menu of "chicken soup, salad, and rice" and its explanation of its relaxing effects.

[1908] Step 6: Generating the ingredient list

[1909] The server creates a list of necessary ingredients based on the generated menu. The input data is the generated menu.

[1910] The server sends a list of ingredients it has created to the user's terminal. The ingredient list includes "chicken, curry powder, various vegetables, and rice."

[1911] Step 7: Use a delivery service

[1912] Users select grocery delivery through the application.

[1913] The device notifies the server of that information.

[1914] The server sends the necessary information (food list and delivery address) to the API of the partnered delivery service, and the delivery process is initiated. Specifically, the server sends the food list to the API, and the package is delivered to the user's address.

[1915] In this way, the system comprehensively considers the user's emotional state, dietary conditions, and special offer information to provide the optimal menu, and further supports the easy acquisition of ingredients through home delivery services.

[1916] (Application Example 2)

[1917] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1918] Existing home cooking support systems offered menu suggestions that took into account users' allergy information and dietary restrictions such as calorie limits, but they lacked the functionality to consider the user's emotional state and suggest menus that responded to it in real time. Furthermore, there is a need to make shopping at physical stores more comfortable and effective by efficiently utilizing sale items and suggesting economical and healthy menus.

[1919] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring the latest sale information, means for acquiring allergy information and dietary conditions such as calorie restrictions entered by the user, emotion recognition means for recognizing the user's emotions, means for generating a menu based on the sale information, the dietary conditions, and the emotion recognition results, means for creating a list of necessary ingredients based on the generated menu, means for placing an order based on the ingredient list and having the ingredients delivered, and means for displaying the generated menu on the user's terminal. This enables menu suggestions that take into account the user's current emotional state and efficient shopping at physical stores.

[1920] "Latest special offer information" refers to special price information for currently available products, obtained from multiple partner retailers.

[1921] "User-entered allergy information" refers to information about a user's allergies that they provide to the system.

[1922] "Calorie restriction" refers to the upper limit of daily calorie intake set by the user.

[1923] "Dietary conditions" refer to specific conditions or restrictions related to diet, such as the user's allergy information or calorie restrictions.

[1924] "Emotion recognition technology" refers to technology that analyzes and recognizes a user's emotions in real time.

[1925] The "means for generating menus" refer to a function that suggests appropriate meal menus based on acquired special offer information, meal conditions, and emotion recognition results.

[1926] The "list of necessary ingredients" is a list of all the ingredients needed to actually prepare the generated menu.

[1927] "A means of placing an order and having ingredients delivered" refers to a function that allows users to purchase ingredients they specify and arrange for them to be delivered to a designated address.

[1928] "User's device" refers to a smartphone, smart glasses, or any other electronic device used by the user.

[1929] A "generative AI model" is an artificial intelligence model that automatically generates menus tailored to user needs based on diverse data.

[1930] To implement this invention, a server, user terminals (smartphones and smart glasses), and API connections from multiple retailers are utilized. The specific hardware and software, data processing, and data calculations required for this are described below.

[1931] Hardware and software

[1932] Server: A computer server installed in a cloud environment.

[1933] User devices: Smartphones, smart glasses (e.g., Google Glass)

[1934] Emotion recognition engine: Microsoft Azure Emotion API

[1935] Database management system: MySQL

[1936] Special Sale Information Acquisition API: APIs for various supermarkets

[1937] Generative AI: OpenAI GPT-3

[1938] Data processing and data calculation

[1939] Emotion recognition and acquisition of sale information

[1940] The device's camera and microphone are used to send user facial recognition data and voice data to an emotion recognition engine. The emotion recognition engine analyzes the user's emotions and displays the results on the device in real time. The server retrieves the latest special offer information from the APIs of partner retailers, receives it in JSON format, and stores it in a database.

[1941] Entering and saving user information

[1942] Upon initial login, users enter allergy information and calorie restrictions via their device. This information is sent to the server and stored in the database.

[1943] Menu generation

[1944] The server retrieves special offer information, user health information, and emotion recognition results from the database. Then, it inputs prompts like the following into the generative AI to generate a menu.

[1945] Example: Prompt text to input to a generative AI

[1946] The user has a dairy allergy and a daily calorie restriction of 1800 calories. The user's current emotional state is fatigue. Please suggest a meal consisting of stir-fried chicken and broccoli, a salad, and whole-wheat bread, all of which are known to help with fatigue recovery.

[1947] The generated menu is saved to a database by the server.

[1948] Menu provision and adjustment

[1949] Users can view the generated menu by wearing smart glasses or using a smartphone app in the store. They can adjust the menu as needed.

[1950] Generating an ingredient list

[1951] Based on the proposed menu, the server extracts the necessary ingredients from the database and displays them on the user's terminal.

[1952] Use of delivery service

[1953] When a user selects a delivery service, the server retrieves the necessary information from the database and sends it to the API of the partnered delivery service. The delivery service then delivers the specified ingredients to the user's address.

[1954] As a concrete example, if a user wears smart glasses and the emotion recognition engine detects "fatigue," the server retrieves "chicken," "broccoli," and "olive oil" as sale items from "Supermarket A." Based on the information the user has entered, such as "dairy allergy" and "1800 calorie limit per day," the generative AI generates a menu of "stir-fried chicken and broccoli, salad, and whole wheat bread." The user confirms this menu, and "chicken, broccoli, olive oil, and whole wheat bread" are displayed as necessary ingredients. If the user selects a delivery service, the ingredients are delivered to their home.

[1955] As a result, users can receive menu suggestions that take their emotional state into consideration and achieve efficient shopping at physical stores.

[1956] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1957] Step 1:

[1958] Users log in to the system via smart glasses or smartphones. After logging in, they enter initial settings such as allergy information and dietary conditions including calorie restrictions. The entered information is sent from the device to the server and stored in the database.

[1959] Input: User allergy information, calorie restriction settings

[1960] Data processing: Information collection and format conversion

[1961] Output: User information stored in the database

[1962] Step 2:

[1963] The user uses the smart glasses' camera and microphone to send facial recognition data and voice data to the emotion recognition engine. The emotion recognition engine analyzes this data and recognizes the user's emotional state in real time. The recognition results are displayed on the device.

[1964] Input: User's facial recognition data, voice data

[1965] Data processing: Data analysis using an emotion recognition engine.

[1966] Output: Real-time emotional state

[1967] Step 3:

[1968] The server retrieves the latest sale information from the APIs of each partner retailer. The retrieved sale information is received in JSON format, the server analyzes the data, and saves the necessary information to the database.

[1969] Input: Special offer information from retailer API

[1970] Data processing: Parsing JSON data and storing it in a database.

[1971] Output: Special sale information stored in the database

[1972] Step 4:

[1973] The server retrieves the user's dietary conditions (allergy information, calorie restrictions), emotion recognition results, and special offer information from the database. Based on this information, it generates prompt messages, which are then input into a generative AI to generate a menu.

[1974] Input: User's dietary conditions, emotional state, special offer information

[1975] Data processing: Prompt processing by generative AI

[1976] Output: Generated menu

[1977] Examples of prompt statements:

[1978] The user has a dairy allergy and a daily calorie restriction of 1800 calories. The user's current emotional state is fatigue. Please suggest a meal consisting of stir-fried chicken and broccoli, a salad, and whole-wheat bread, all of which are known to help with fatigue recovery.

[1979] Step 5:

[1980] The server sends the generated menu to the user's terminal. The user can review the menu and make adjustments as needed. The adjustment results are then sent back to the server.

[1981] Input: Generated menu

[1982] Data processing: Information transmission and user adjustments.

[1983] Output: Adjusted final menu

[1984] Step 6:

[1985] The server creates a list of necessary ingredients based on the final menu and sends it to the terminal. The user reviews the list and selects the ingredients they want to order.

[1986] Input: Final Menu

[1987] Data processing: Extraction and creation of a list of necessary ingredients.

[1988] Output: Ingredient list

[1989] Step 7:

[1990] When a user selects grocery delivery, the device notifies the server of this information. The server then sends the necessary information to the API of its partner delivery service and initiates the process of delivering the groceries to the user's address.

[1991] Input: Ingredient list, delivery selection information

[1992] Data processing: Sending data to delivery service APIs

[1993] Output: Food items delivered to the user's address

[1994] The above outlines the specific processing steps of the program of this invention. Through these steps, users can achieve healthier and more efficient shopping based on personalized suggestions that take their emotional state into consideration.

[1995] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1996] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1997] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1998] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1999] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[2000] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[2001] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[2002] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[2003] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[2004] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[2005] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[2006] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[2007] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[2008] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2009] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[2010] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[2011] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[2012] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[2013] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[2014] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[2015] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[2016] The following is further disclosed regarding the embodiments described above.

[2017] (Claim 1)

[2018] Means of obtaining the latest sale information,

[2019] A means of obtaining allergy information and dietary conditions such as calorie restrictions entered by the user,

[2020] A means for generating a menu based on the aforementioned special offer information and the aforementioned meal conditions,

[2021] A means for creating a list of necessary ingredients based on the generated menu,

[2022] A means of placing an order based on the aforementioned list of ingredients and delivering the ingredients,

[2023] A system that includes this.

[2024] (Claim 2)

[2025] The system according to claim 1, which obtains special sale information from multiple affiliated retailers.

[2026] (Claim 3)

[2027] The system according to claim 1, which displays the generated menu on the user's terminal.

[2028]

[2029] "Example 1"

[2030] (Claim 1)

[2031] A means of gathering the latest sale information,

[2032] A means of obtaining allergy information and dietary conditions such as calorie restrictions entered by the user,

[2033] A means for generating a menu using an AI model based on the aforementioned special sale information and the aforementioned meal conditions,

[2034] A means for creating a list of necessary ingredients based on the generated menu,

[2035] A means of placing an order based on the aforementioned list of ingredients and delivering the ingredients,

[2036] A system that includes this.

[2037] (Claim 2)

[2038] The system according to claim 1, which obtains special sale information from multiple affiliated retail stores.

[2039] (Claim 3)

[2040] The system according to claim 1, which displays the generated menu on the user's terminal.

[2041] "Application Example 1"

[2042] (Claim 1)

[2043] Means of obtaining the latest sale information,

[2044] A means of obtaining allergy information and dietary conditions such as calorie restrictions entered by the user,

[2045] A means of using a generative AI model that generates menus based on the aforementioned special sale information and the aforementioned meal conditions,

[2046] A means for creating a list of necessary ingredients based on the generated menu,

[2047] A means of placing an order based on the aforementioned list of ingredients and delivering the ingredients,

[2048] A system that includes this.

[2049] (Claim 2)

[2050] The system according to claim 1, which obtains special sale information from multiple affiliated retailers.

[2051] (Claim 3)

[2052] The system according to claim 1, which displays the generated menu on the user's terminal.

[2053] "Example 2 of combining an emotion engine"

[2054] (Claim 1)

[2055] Means of obtaining the latest sale information,

[2056] A means of obtaining allergy information and dietary conditions such as calorie restrictions entered by the user,

[2057] A means of recognizing the user's emotional state,

[2058] Means for using a generative AI model that generates a menu based on the aforementioned special offer information, the aforementioned meal conditions, and the aforementioned emotional state,

[2059] A means for creating a list of necessary ingredients based on the generated menu,

[2060] A means of placing an order based on the aforementioned list of ingredients and delivering the ingredients,

[2061] A system that includes this.

[2062] (Claim 2)

[2063] The system according to claim 1, which obtains special sale information from multiple affiliated retailers.

[2064] (Claim 3)

[2065] The system according to claim 1, which displays the generated menu on the user's terminal.

[2066] "Application example 2 when combining with an emotional engine"

[2067] (Claim 1)

[2068] Means of obtaining the latest sale information,

[2069] A means of obtaining allergy information and dietary conditions such as calorie restrictions entered by the user,

[2070] A means of recognizing the user's emotions,

[2071] A means for generating a menu based on the aforementioned special offer information, the aforementioned meal conditions, and the aforementioned emotion recognition results,

[2072] A means for creating a list of necessary ingredients based on the generated menu,

[2073] A means of placing an order based on the aforementioned list of ingredients and delivering the ingredients,

[2074] A means for displaying the generated menu on the user's terminal,

[2075] A system that includes this.

[2076] (Claim 2)

[2077] The system according to claim 1, which obtains special sale information from multiple affiliated retailers.

[2078] (Claim 3)

[2079] The system according to claim 1, which uses a generative AI model to generate a menu adjusted based on the user's real-time emotional state. [Explanation of Symbols]

[2080] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means of obtaining the latest sale information, A means of obtaining allergy information and dietary conditions such as calorie restrictions entered by the user, A means for generating a menu based on the aforementioned special offer information and the aforementioned meal conditions, A means for creating a list of necessary ingredients based on the generated menu, A means of placing an order based on the aforementioned list of ingredients and delivering the ingredients, A system that includes this.

2. The system according to claim 1, which obtains special sale information from multiple affiliated retailers.

3. The system according to claim 1, which displays the generated menu on the user's terminal.

Images