System

Abstract

A system is provided.SOLUTION: A system, comprising: means for a user to input ingredients; means for a terminal to send a list of ingredients input by the user to a server; means for the server to generate a recipe using a generative model; means for the server to send the generated recipe to the terminal; and means for the terminal to display the generated recipe to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's busy lifestyles, cooking different, delicious meals every day can be a challenge for many people. In particular, many people often end up wasting food because they don't know how to use the ingredients they have on hand. It can also be difficult to run out of recipe ideas or find recipes that suit their tastes or health conditions. As a result, there is a constant need to come up with new recipe ideas, expand the variety of daily meals, and minimize food waste.

[0005] The purpose of this invention is to solve these problems by enabling users to easily create new recipes using ingredients available at home and enjoy delicious cooking, while also providing recipes that take into consideration the user's preferences and health management. [Means for solving the problem]

[0006] The present invention solves the above problems by the following means.

[0007] The system includes a means for a user to input ingredients, a means for a terminal to transmit the ingredient list input by the user to a server, a means for the server to generate a recipe using a generative model, a means for the server to transmit the generated recipe to the terminal, and a means for the terminal to display the generated recipe to the user.

[0008] Furthermore, the recipe generation means can adjust the parameters of the generation model according to the user's preferences, and it is also possible to generate recipes specialized for health management. This allows users to constantly obtain new cooking ideas, reduce food waste, and enjoy recipes that suit their preferences and health.

[0009] "User" refers to an individual who uses this system to input ingredients and view recipes.

[0010] "Terminal" refers to a device that allows a user to input ingredients and view the generated recipe. Specifically, this refers to an electronic device such as a smartphone, tablet, or PC.

[0011] "Server" refers to a computer system that receives an ingredient list, generates a recipe using a generative AI model, and sends the generated results to a terminal.

[0012] "Generative AI model" refers to the artificial intelligence algorithm used to generate recipes based on an input list of ingredients.

[0013] "Ingredients" refers to ingredients for a dish entered by the user, specifically foods such as chicken, carrots, and potatoes.

[0014] The "ingredient list" refers to a list of multiple ingredients entered by the user.

[0015] A "recipe" refers to a series of information about the steps, ingredients, and cooking methods for creating a dish using ingredients.

[0016] "Preferences" refers to the user's individual taste preferences and recipe selection criteria.

[0017] "Health management" refers to an index for creating recipes that takes into account the user's health condition and nutritional balance.

[0018] An "HTTP request" refers to a communication message sent by a terminal to a server to request information.

[0019] An "HTTP response" refers to a communication message that a server sends to return information in response to a request from a terminal.

[0020] These definitions clarify the function of the entire system and the role of each element. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] This invention relates to a system that generates and provides new recipes using ingredients that a user has at home when the user inputs them into an app. Specifically, the system includes a means for the user to input ingredients, a means for the device to send the input ingredient list to a server, a means for the server to generate a recipe using a generative AI model, a means for sending the generated recipe to the device, and a means for the device to display the generated recipe to the user.

[0043] System program processing overview

[0044] The system is programmed as follows:

[0045] Ingredients input stage

[0046] The user launches the app and displays the ingredient input screen. They enter the ingredients they want to use in the text box and tap the "Add" button to add them to the ingredient list. After entering all the ingredients, they tap the "Submit" button.

[0047] Ingredient list sending stage

[0048] The device sends the ingredient list entered by the user to the server. The device converts the ingredient list into JSON format, stores it in the HTTP request body, and sends it to the server.

[0049] Recipe generation stage by the server

[0050] The server receives and analyzes the ingredient list. The server passes the ingredient list as input to the generative AI model to generate a recipe. The generated recipe is formatted in JSON format and sent to the device as an HTTP response.

[0051] Recipe display stage

[0052] The terminal analyzes the response received from the server and acquires the recipe information. The terminal displays the recipe screen and provides the generated recipe to the user.

[0053] Specific examples

[0054] For example, if a user has chicken, carrots, and potatoes at home, a new recipe will be generated using those ingredients.

[0055] The user inputs ingredients into the app, sequentially entering "chicken," "carrots," and "potatoes." When the user taps the "Send" button, the device sends the list of ingredients to the server.

[0056] The server analyzes the received ingredient list and inputs it into a generative AI model. The generative AI model generates a recipe such as "chicken and vegetable stew," and the server formats the recipe information and sends it to the device.

[0057] The device receives this recipe information and shows the recipe display screen to the user, who can then start cooking based on the recipe for "chicken and vegetable stew."

[0058] In this way, this system helps users efficiently use ingredients on hand and enjoy new dishes. At the same time, it reduces food waste and increases the variety of daily dishes. It is also possible to customize recipes generated according to the user's preferences and generate recipes specialized for health management.

[0059] The processing flow will be explained below.

[0060] Step 1:

[0061] The user launches the app and the app's home screen appears.

[0062] Step 2:

[0063] The user taps the "Create a new recipe" button. The ingredient input screen appears.

[0064] Step 3:

[0065] The user enters the ingredients they want to use in the text box, entering "chicken," "carrots," and "potatoes" in that order, and taps the "Add" button to add them to the ingredients list.

[0066] Step 4:

[0067] The user enters all ingredients and taps the "Submit" button. The device retrieves the ingredient list and proceeds to the next step.

[0068] Step 5:

[0069] The device converts the ingredient list into JSON format and stores the JSON data of the ingredient list in the HTTP request body.

[0070] Step 6:

[0071] The device sends an HTTP request containing JSON data to the server. The request URL is used to send the data to the server.

[0072] Step 7:

[0073] The server receives the HTTP request and extracts the JSON data of the ingredient list from the request body.

[0074] Step 8:

[0075] The server parses the JSON data of the ingredient list and obtains information about each ingredient: "chicken," "carrot," and "potato."

[0076] Step 9:

[0077] The server passes the list of ingredients as input to the generative AI model, which then generates a recipe based on the ingredients.

[0078] Step 10:

[0079] The server converts the generated recipe information into JSON format, generates an HTTP response, and stores the JSON recipe information in the response body.

[0080] Step 11:

[0081] The server sends an HTTP response including the generated recipe information to the terminal.

[0082] Step 12:

[0083] The device receives the HTTP response and obtains the recipe information JSON data from the response body.

[0084] Step 13:

[0085] The device parses the JSON data and extracts the recipe information for "Chicken and Vegetable Stew." It then generates the recipe screen.

[0086] Step 14:

[0087] The device displays a recipe viewing screen, providing the user with information such as the recipe title, ingredients, and cooking steps.

[0088] Step 15:

[0089] The user checks the displayed recipes and starts cooking based on the recipe "Chicken and Vegetable Stew."

[0090] These are the specific processing steps of this system, which allows users to efficiently use ingredients at home and obtain new cooking recipes.

[0091] Example 1

[0092] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0093] In today's busy lifestyles, users need a way to efficiently utilize ingredients on hand and easily create new recipes. However, conventional systems have complicated processes for inputting ingredients and creating recipes, which is time-consuming for users. In addition, they lack the functionality to create recipes specialized for health management or customized recipes based on user preferences.

[0094] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0095] In this invention, the server includes a means for analyzing the ingredient list and converting it into a prompt statement, a means for inputting the prompt statement into a generative AI model to generate a recipe, and a means for formatting the generated recipe into JSON format and sending it to the terminal as an HTTP response. This allows the user to easily input ingredients on hand and automatically generate a new recipe using the generative AI model. It also simultaneously enables parameter adjustment according to the user's preferences and the generation of recipes specialized for health management.

[0096] 1. A "user" is an individual who uses the system to input ingredients on hand and generate recipes.

[0097] 2. "Terminal" means the collection of hardware and software used by a user to input ingredients, transmit the input to the server, and receive and display the generated recipe.

[0098] 3. "Server" means a collection of hardware and software that analyzes the ingredient list received from the Terminal, generates a recipe using a generative AI model, and sends it to the Terminal.

[0099] 4. An "ingredient list" is a collection of data that a user enters to identify the ingredients they wish to use.

[0100] 5. "JSON format" stands for JavaScript Object Notation and is a standard for representing data in a lightweight text format.

[0101] 6. "HTTP request" is a part of the communication protocol used by a device to send data to a server.

[0102] 7. "HTTP response" is part of the communication protocol used by the server to send data to the terminal.

[0103] 8. A "prompt" is a sentence that is input to the generative AI model to generate a recipe based on a list of ingredients.

[0104] 9. “Generative AI Model” means an artificial intelligence model for generating new recipes based on input prompts.

[0105] 10. "Recipe display screen" is an interface that allows the terminal to visually display to the user recipe information received from the server.

[0106] The above definitions of each word concretely show the functions and roles of the system.

[0107] MODE FOR CARRYING OUT THE INVENTION

[0108] An embodiment of this invention is a system that generates and provides new recipes using ingredients when a user inputs ingredients available at home into an application. Specifically, the system includes a means for the user to input ingredients, a means for the terminal to send the input ingredient list to a server, a means for the server to generate a recipe using a generative AI model, a means for sending the generated recipe to the terminal, and a means for the terminal to display the generated recipe to the user.

[0109] Hardware and software used

[0110] Hardware

[0111] Devices such as smartphones, tablets, and computers

[0112] Server for recipe generation and data processing

[0113] software

[0114] Application for inputting ingredients

[0115] Data communication using the HTTP protocol

[0116] Generative AI models (e.g., OpenAI GPT-3)

[0117] System configuration and operation

[0118] Below, each component of this system and its operation will be explained in detail.

[0119] The user launches the application on their smartphone or tablet and displays the ingredient input screen. The user enters the ingredients they want to use one by one into the text box and taps the "Add" button to add them to the ingredient list. After entering all the ingredients, the user taps the "Submit" button.

[0120] The device obtains the ingredient list entered by the user and converts it to JSON format. The converted JSON data is stored in the body of the HTTP request and an HTTP POST request is sent to the specified endpoint on the server. This request includes the ingredient list entered by the user.

[0121] The server receives the HTTP request and extracts the ingredient list from the request body. It then parses the ingredient list to generate a prompt. For example, it generates a prompt like, "The user has the following ingredients: - chicken - carrots - potatoes. Please generate a new recipe using these ingredients."

[0122] The generated prompt is input into a generative AI model to generate a recipe. The generative AI model generates a new recipe (e.g., "chicken and vegetable stew") and formats the result in JSON format. The formatted JSON data is stored in the body of an HTTP response and sent to the device.

[0123] The device receives the HTTP response from the server and analyzes the JSON data in the response. Based on the analysis results, it creates a recipe display screen and displays the new recipe to the user. The user can then start cooking according to the recipe.

[0124] Specific examples

[0125] For example, if a user has chicken, carrots, and potatoes at home, a new recipe will be generated using those ingredients.

[0126] The user launches the application, enters "chicken," "carrots," and "potatoes," and taps the "Send" button. The device sends the ingredient list to the server, which analyzes the received ingredient list and inputs it into the generative AI model. The generative AI model generates a recipe (e.g., "chicken and vegetable stew"), and the server formats the recipe information in JSON format and sends it to the device. The device receives this recipe information and displays the recipe display screen to the user. The user can then start cooking based on the "chicken and vegetable stew" recipe.

[0127] In this way, users can efficiently use ingredients they have on hand and enjoy new dishes. In addition, by adjusting the parameters of the generative AI model, it is possible to customize recipes according to user preferences and generate recipes specialized for health management.

[0128] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0129] The flow of this system's program processing

[0130] Step 1:

[0131] The user launches the application and the ingredient input screen is displayed. The user enters the ingredients they want to use one by one into the text box and taps the "Add" button. For example, the user can enter "chicken," "carrot," and "potato" in that order. When the user taps the "Submit" button, all the ingredients entered are included in the list.

[0132] Input: User input of ingredients (e.g. "chicken", "carrot", "potato")

[0133] Output: List of ingredients (e.g. ["chicken", "carrot", "potato"])

[0134] Step 2:

[0135] The device converts the ingredient list entered by the user into JSON format and stores it in the body of the HTTP request. This request is then sent to the server. Specifically, the ingredient list is sent to the specified endpoint on the server with the "application / json" content type.

[0136] Input: List of ingredients (e.g. ["chicken", "carrot", "potato"])

[0137] Output: JSON format data and HTTP request (e.g., {"ingredients":["chicken", "carrot", "potato"]})

[0138] Step 3:

[0139] The server analyzes the received HTTP request and retrieves the list of ingredients from the request body. It then uses the retrieved ingredient list to generate a prompt. This prompt is input into the generative AI model and serves as an instruction for generating a recipe. For example, the prompt might be in the form, "The user has the following ingredients: - chicken - carrots - potatoes. Please generate a new recipe using these ingredients."

[0140] Input: HTTP request, JSON format data (e.g., {"ingredients":["chicken", "carrot", "potato"]})

[0141] Output: Prompt text (e.g., "The user has the following ingredients: - chicken - carrots - potatoes. Generate a new recipe using these ingredients.")

[0142] Step 4:

[0143] The server inputs the prompt into a generative AI model to generate a new recipe. A generative AI model (e.g., OpenAI GPT-3) is used to generate a recipe based on the prompt. The generated recipe is then converted back to JSON format and stored in the body of the HTTP response.

[0144] Input: Prompt text (e.g., "The user has the following ingredients: - chicken - carrots - potatoes. Generate a new recipe using these ingredients.")

[0145] Output: Recipe information (e.g., "Chicken and vegetable stew"), JSON-formatted recipe data (e.g., {"Recipe": "Chicken and vegetable stew", "Steps": ["Cut the chicken", "Chop the vegetables", "Simmer"]})

[0146] Step 5:

[0147] The server stores the JSON data of the generated recipe in the HTTP response body and sends it to the terminal.

[0148] Input: JSON-formatted recipe data (e.g., {"Recipe": "Chicken and vegetable stew", "Steps": ["Cut the chicken", "Chop the vegetables", "Simmer"]})

[0149] Output: HTTP response, recipe data in JSON format

[0150] Step 6:

[0151] The device receives the HTTP response from the server and retrieves the JSON-formatted recipe data from the response body. It then analyzes this data to create a recipe display screen and presents it to the user. The user can then check the recipe and start cooking.

[0152] Input: HTTP response, JSON-formatted recipe data (e.g., {"Recipe": "Chicken and vegetable stew", "Steps": ["Cut the chicken", "Chop the vegetables", "Simmer"]})

[0153] Output: Recipe display screen, visual display of recipe information

[0154] The above is the flow of processing in the program of this system and the specific operations performed at each step.

[0155] (Application example 1)

[0156] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0157] While existing systems exist that efficiently utilize ingredients available at home to generate new recipes, they do not include a means for easily obtaining ingredients or seasonings that are in short supply, which means that users have to go shopping separately to obtain ingredients that are not available at home, resulting in inconvenience. This also leads to food waste and limits the variety of dishes that can be made. Therefore, there is a need for the development of a new system that allows users to easily obtain ingredients and seasonings that are in short supply.

[0158] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0159] In this invention, the server includes a means for inputting ingredients, a means for transmitting an ingredient list to the server, a means for generating a recipe using a generative model, a means for transmitting the generated recipe to a terminal, a means for displaying the generated recipe, and a means for the user to order delivery of ingredients and seasonings that the user is running low on. This allows the user to make the most of ingredients they have on hand and efficiently obtain ingredients and seasonings that they are running low on, enabling them to enjoy new variations of dishes.

[0160] "User" refers to an individual who uses this system to input ingredients, create recipes, and place delivery orders.

[0161] "Terminal" refers to an electronic device used by a user, such as a smartphone or computer.

[0162] The "ingredient list" refers to a list of multiple ingredients entered by the user.

[0163] "Server" refers to a computer system that runs on the cloud or a network and generates recipes and communicates data.

[0164] A "generative model" refers to a machine learning algorithm or AI system that creates new recipes based on input data.

[0165] A "recipe" refers to detailed information about cooking steps and ingredients generated using an ingredient list.

[0166] "Delivery order" refers to the act of a user ordering ingredients or seasonings they are running low on via the Internet and having them delivered to their home.

[0167] "Display means" refers to a function for visually showing the recipe generated on the terminal to the user.

[0168] The present invention provides a system that allows a user to make the most of ingredients on hand and efficiently obtain ingredients and seasonings that are in short supply. Specific embodiments of the present invention are described below.

[0169] System Configuration

[0170] The system includes a terminal used by a user (e.g., a smartphone or tablet), a server, and a network that communicates via the Internet.

[0171] Hardware and software used

[0172] Device: Smartphone or tablet (device where user inputs ingredients and displays recipe and delivery order information).

[0173] Server: A computer system in the cloud (implemented using Python and Flask).

[0174] Generative AI models: Generative models such as OpenAI GPT-4.

[0175] Network: A network for transmitting data between devices and servers over the Internet.

[0176] Function and process description

[0177] Enter ingredients

[0178] The user launches the app on their device and inputs the ingredients they have on hand. On the ingredient input screen, the user enters the ingredients they want to use in the text box and taps the "Add" button to add them to the ingredient list.

[0179] Send ingredient list

[0180] The terminal converts the ingredient list entered by the user into JSON format, stores it in the HTTP request body, and sends it to the server.

[0181] Recipe Generation

[0182] The server receives the input ingredient list and analyzes it. The analyzed ingredient list is input into the generative AI model, and a new recipe is generated. The recipe generated by the generative AI model is formatted in JSON and sent to the device as an HTTP response.

[0183] View recipes and order delivery

[0184] The device analyzes the response received from the server and obtains the generated recipe information. The user can then cook based on this information. They can also order delivery of missing ingredients and seasonings based on the generated recipe.

[0185] Specific examples

[0186] For example, if a user has "tomatoes," "olive oil," and "basil" at home, the following processing will occur:

[0187] Example prompt sentence:

[0188] "Suggest a new recipe using the following ingredients: tomatoes, olive oil, and basil."

[0189] Generated recipe example:

[0190] Tomato and Basil Pasta: Finely chop the tomatoes and fry them in olive oil. Add the basil and fry some more, then mix with the cooked pasta. Season with salt and pepper and top with Parmesan cheese.

[0191] The user can start cooking based on this generated recipe information, and can easily order delivery to get any ingredients they are missing.

[0192] This allows users to make the most of the ingredients they have on hand, expanding the variety of dishes they can make, while also reducing food waste.

[0193] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0194] Step 1:

[0195] Enter ingredients

[0196] The user launches the app on their device and displays the ingredient input screen. The user types the ingredients they have at home into the text box and taps the "Add" button. This adds the ingredients to the list. After entering all the ingredients, the user taps the "Submit" button.

[0197] Input: Ingredients entered by the user.

[0198] Output: A list of ingredients is generated.

[0199] Step 2:

[0200] Send ingredient list

[0201] The device converts the ingredient list entered by the user into JSON format, then sends this JSON data to the server as an HTTP POST request.

[0202] Input: A list of ingredients entered by the user.

[0203] Output: A JSON formatted list of ingredients is sent to the server.

[0204] Step 3:

[0205] Recipe Generation

[0206] The server parses the JSON-formatted ingredient list received from the device. The server calls the API of a generative AI model (e.g., OpenAI GPT-4) and provides the ingredient list as input data. The generative AI model generates a new recipe based on the input data. The generated recipe is formatted in JSON format.

[0207] Input: A list of ingredients in JSON format.

[0208] Output: The new recipe generated by the generative AI model.

[0209] Step 4:

[0210] Send recipe

[0211] The server sends the created recipe to the terminal as an HTTP response in JSON format.

[0212] Input: The generated recipe.

[0213] Output: The recipe in JSON format is sent to the terminal.

[0214] Step 5:

[0215] Recipe display

[0216] The device analyzes the JSON response received from the server. The device displays the analyzed recipe information on the recipe screen. The user can check the displayed recipe and start cooking. It is also possible to order delivery of missing ingredients and seasonings based on the recipe.

[0217] Input: Recipe information in JSON format.

[0218] Output: The parsed recipe information is displayed to the user.

[0219] Through these steps, users can not only efficiently utilize the ingredients they have on hand and create new recipes, but also easily order delivery of ingredients or seasonings they are running low on.

[0220] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0221] This invention relates to a system that generates and provides new recipes using ingredients when a user inputs ingredients available at home into an application. Specifically, the system includes a means for the user to input ingredients, a means for the terminal to send the input ingredient list to a server, a means for the server to generate a recipe using a generative model, a means for the terminal to send the generated recipe, a means for the terminal to display the generated recipe to the user, and an emotion engine that recognizes the user's emotions and adjusts the parameters of a recipe generation model based on the emotions.

[0222] System program processing overview

[0223] The system is programmed as follows:

[0224] Ingredients input stage

[0225] The user launches the app and the ingredient input screen is displayed. The user enters the ingredients they want to use and taps the Add button to add them to the ingredient list. After entering all ingredients, they tap the Send button.

[0226] Ingredient list sending stage

[0227] The device sends the ingredient list entered by the user to the server. The device converts the ingredient list into JSON format, stores it in the HTTP request body, and sends it to the server.

[0228] Emotion recognition stage by server

[0229] The server receives the HTTP request and extracts the JSON data of the ingredient list from the request body. Next, the emotion engine recognizes the user's emotion. Emotions are acquired and analyzed using sensor data from cameras, microphones, etc.

[0230] Recipe Generation Stage

[0231] The server analyzes the ingredient list and adjusts the parameters of the generative AI model based on the user's perceived emotions. For example, if the user is feeling stressed, the model is adjusted to prioritize ingredients and cooking methods that have a relaxing effect. The server then inputs the ingredient list into the generative AI model to generate a recipe.

[0232] Recipe submission stage

[0233] The server converts the generated recipe information into JSON format and sends it to the terminal as an HTTP response.

[0234] Recipe display stage

[0235] The terminal analyzes the response received from the server and acquires the recipe information. The terminal displays a recipe viewing screen and provides the created recipe to the user.

[0236] Specific examples

[0237] For example, if a user wants to get new recipes using "chicken," "carrots," and "potatoes," he or she will use this system.

[0238] The user inputs ingredients into the app, entering "chicken," "carrots," and "potatoes" respectively. When the user taps the send button, the device sends the list of ingredients to the server.

[0239] The server analyzes the received ingredient list and also recognizes the user's emotions from camera and microphone data. For example, if the server recognizes that the user is tired, the generative AI model generates a recipe with an emphasis on "dishes that have a refreshing effect." The generated recipe is "chicken and carrot soup."

[0240] The server converts this recipe information into JSON format and sends it to the device.

[0241] The device receives the recipe information and shows the recipe display screen to the user, who then begins cooking based on the recipe for "chicken and carrot soup."

[0242] In this way, the system can generate optimal recipes based on the user's emotions, improving the user's cooking experience by providing new dishes without wasting ingredients on hand.

[0243] The processing flow will be explained below.

[0244] Step 1:

[0245] The user launches the app and the home screen appears.

[0246] Step 2:

[0247] The user taps the "Create a new recipe" button. The ingredient input screen appears.

[0248] Step 3:

[0249] The user enters the ingredients they want to use in the text box, entering "chicken," "carrots," and "potatoes" in that order, and taps the "Add" button to add them to the ingredients list.

[0250] Step 4:

[0251] The user enters all ingredients and taps the "Submit" button. The device retrieves the ingredient list and proceeds to the next step.

[0252] Step 5:

[0253] The device converts the ingredient list into JSON format and stores the JSON data of the ingredient list in the HTTP request body.

[0254] Step 6:

[0255] The device sends an HTTP request containing JSON data to the server. The request URL is used to send the data to the server.

[0256] Step 7:

[0257] The server receives the HTTP request and extracts the JSON data of the ingredient list from the request body.

[0258] Step 8:

[0259] The server parses the JSON data of the ingredient list and obtains information about each ingredient: "chicken," "carrot," and "potato."

[0260] Step 9:

[0261] The server activates the emotion engine to recognize the user's emotions. It acquires sensor data from cameras and microphones and analyzes the user's facial expressions and tone of voice.

[0262] Step 10:

[0263] The server determines the user's emotion from the analysis results of the emotion engine. For example, it determines that the user is tired.

[0264] Step 11:

[0265] The server adjusts the parameters of the generative AI model based on the analysis results of the emotion engine, for example, by changing the settings to prioritize dishes with a refreshing effect.

[0266] Step 12:

[0267] The server passes the list of ingredients as input to the generative AI model and generates a recipe, for example, "chicken and carrot soup."

[0268] Step 13:

[0269] The server converts the generated recipe information into JSON format, generates an HTTP response, and stores the JSON recipe information in the response body.

[0270] Step 14:

[0271] The server sends an HTTP response including the generated recipe information to the terminal.

[0272] Step 15:

[0273] The device receives the HTTP response and obtains the recipe information JSON data from the response body.

[0274] Step 16:

[0275] The device parses the JSON data and extracts the recipe information for "Chicken and Carrot Soup." It then generates the recipe screen.

[0276] Step 17:

[0277] The device displays a recipe viewing screen, providing the user with information such as the recipe title, ingredients, and cooking steps.

[0278] Step 18:

[0279] The user checks the displayed recipes and starts cooking based on the "Chicken and Carrot Soup" recipe.

[0280] These are the specific processing steps in this system, which allows users to obtain the best recipes based on the ingredients and emotions at home.

[0281] Example 2

[0282] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0283] Conventional recipe generation systems are unable to generate recipes that reflect the user's current emotions and preferences, and the recipes they provide may not meet the user's expectations or needs. Furthermore, they have issues with being unable to efficiently utilize the ingredients the user has on hand, and not proposing recipes that are appropriate from a health management perspective.

[0284] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for recognizing a user's emotion using an emotion recognition engine, a means for adjusting parameters of a generative AI model based on the recognized emotion and generating a recipe, and a means for transmitting the generated recipe to a terminal. This makes it possible to provide an optimal recipe according to the user's emotion and preferences.

[0285] "User" refers to a person who uses the system to input ingredients and obtain recipes.

[0286] A "terminal" is a device that allows a user to input ingredients, transmits an ingredient list to a server, and displays the recipe received from the server.

[0287] The "server" is a computer system that receives the ingredient list sent by the user, recognizes the user's emotions using an emotion recognition engine, generates a recipe using a generative AI model, and sends the generated recipe to the terminal.

[0288] An "emotion recognition engine" is a system or algorithm that analyzes data obtained from the user's camera or microphone to recognize the user's emotions.

[0289] A "generative AI model" is a machine learning model that automatically generates new recipes based on input data, and is particularly capable of adjusting parameters according to the user's emotions.

[0290] The "ingredient list" is a list of multiple ingredients that the user wants to use, and is data in a format that is sent from the terminal to the server.

[0291] A "recipe" is information that provides detailed instructions for making a dish using ingredients, and is generated by a generative AI model.

[0292] "Parameter adjustment" refers to changing the internal settings of a generative AI model to change its behavior depending on specific conditions or situations.

[0293] This invention relates to a system that generates and provides new recipes using ingredients that a user has at home when they input them into an application. This system is composed of the following specific hardware and software.

[0294] Hardware and software used

[0295] The terminal used by the user is a mobile device such as a smartphone or tablet, and an application for inputting ingredients is installed on this terminal.

[0296] The server is a typical server providing cloud computing services, and includes web server software (e.g., Apache or Nginx) to process HTTP requests, software to run a generative AI model (e.g., GPT-4) that analyzes ingredient lists and generates recipes, and image and voice analysis technologies required to run an emotion recognition engine.

[0297] Examples of user actions

[0298] When a user launches the app, an ingredient input screen appears. This screen has a text box and an "Add" button and a "Submit" button. The user inputs ingredients such as "chicken," "carrots," and "potatoes," and taps the "Add" button to add them to the ingredient list. Once all ingredients have been input, the user taps the "Submit" button to confirm the list.

[0299] Specific examples of terminal operation

[0300] The device sends the ingredient list entered by the user to the server. At this time, the ingredient list is converted to JSON format and sent to the server as the body of the HTTP request.

[0301] Example of server operation

[0302] When the server receives an HTTP request, it extracts the JSON data from the request body, parses the list of ingredients, and uses an emotion recognition engine to recognize the user's emotion. For example, it can recognize the user's emotion of "tired" through sensor data collected from the camera and microphone.

[0303] Next, the server adjusts the parameters of the generative AI model based on the recognized user's emotions and generates a recipe. For example, a prompt commanding the model to generate a "refreshing recipe for a tired user" is input. Suppose the generated recipe is "chicken and carrot soup."

[0304] "Please suggest a refreshing dish using chicken, carrots, and potatoes for tired users."

[0305] Use a prompt such as:

[0306] The server converts the generated recipe information into JSON format and sends it to the terminal as an HTTP response. The terminal analyzes the received response and obtains the generated recipe information (e.g., "Chicken and Carrot Soup"). Finally, the terminal displays the recipe viewing screen to the user.

[0307] In this way, the system can provide optimal recipes based on the user's emotions and preferences, providing a new cooking experience without wasting the ingredients the user has.

[0308] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0309] Step 1:

[0310] The user launches the app and displays the ingredient input screen.

[0311] Specific operation: The user taps the app icon on their smartphone or tablet to launch the application. After launching, the ingredient input screen is displayed.

[0312] Input and output: The input is a user operation (tapping the app icon), and the output is the display of the ingredient input screen.

[0313] Step 2:

[0314] The user enters the ingredients they want to use and taps the add button.

[0315] Specific behavior: The user enters "chicken" in the text box and taps the "Add" button. This displays the ingredients in the text box as a list. This operation is repeated for "carrots" and "potatoes."

[0316] Input and output: The input is the ingredient name and tapping the add button, and the output is the display of the ingredient list.

[0317] Step 3:

[0318] After the user enters all the ingredients, he taps the submit button.

[0319] Specific actions: Once the user has entered all the required ingredients, they tap the "Submit" button on the screen.

[0320] Input and Output: The input is the tap of the send button, and the output is the trigger of the send request.

[0321] Step 4:

[0322] The terminal transmits the ingredient list input by the user to the server.

[0323] Specific operation: The device serializes the input ingredient list into JSON format, stores it in the HTTP request body, and sends it to the server.

[0324] Input and Output: The input is the list of ingredients entered by the user, and the output is the HTTP request sent to the server.

[0325] Step 5:

[0326] The server receives the HTTP request and extracts the JSON data of the ingredient list from the request body.

[0327] Specific behavior: The server parses the received HTTP request and extracts a JSON-formatted list of ingredients from the request body.

[0328] Input and Output: The input is the HTTP request sent from the terminal, and the output is the parsed JSON data of the ingredient list.

[0329] Step 6:

[0330] The server uses an emotion recognition engine to recognize the user's emotion.

[0331] Specific operation: The server analyzes sensor data obtained from the camera and microphone and recognizes the user's emotions. For example, it detects emotions such as "tired" from the user's facial expression and tone of voice.

[0332] Input and Output: The input is the sensor data and the ingredient list, and the output is the recognized user emotion data.

[0333] Step 7:

[0334] The server adjusts the parameters of the generative AI model based on the recognized emotions and generates a recipe.

[0335] Specific operation: The server generates a prompt sentence based on the emotion recognition result and inputs it into a generative AI model (e.g., GPT-4). Let's assume that the generated recipe is "chicken and carrot soup."

[0336] Input and Output: The input is the recognized emotion data and the ingredients list, and the output is the generated recipe.

[0337] Step 8:

[0338] The server converts the generated recipe information into JSON format and sends it to the terminal as an HTTP response.

[0339] Specific operation: The server serializes the generated recipe information in JSON format, stores it in the body of the HTTP response, and sends it to the terminal.

[0340] Input and Output: The input is the generated recipe information, and the output is the HTTP response sent to the terminal.

[0341] Step 9:

[0342] The terminal analyzes the response received from the server and acquires the recipe information.

[0343] Specific operation: The device parses the HTTP response received and extracts the recipe information.

[0344] Input and Output: The input is the HTTP response received from the server, and the output is the extracted recipe information.

[0345] Step 10:

[0346] The terminal displays a recipe viewing screen and provides the generated recipe to the user.

[0347] Specific operation: The device displays a recipe viewing screen and provides the user with detailed information such as ingredients, steps, and tips for the dish.

[0348] Input and Output: The input is the extracted recipe information and the output is the recipe screen that is displayed to the user.

[0349] This system allows users to efficiently use ingredients they have on hand and obtain the optimal recipe based on their emotional state.

[0350] (Application example 2)

[0351] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0352] Conventional recipe generation systems only generate recipes based on the ingredients a user has on hand, and do not consider the user's emotions or mood when suggesting recipes. Furthermore, they lack a function for instantly ordering food based on the generated recipe, forcing users to search for purchasing locations and ordering methods themselves based on the suggested recipe. This results in a poor user experience and a lack of convenience. To solve these problems, a system is needed that generates recipes that consider the user's emotions and seamlessly processes food orders from external organizations.

[0353] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0354] In this invention, the server includes a means for recognizing a user's emotion, a means for adjusting parameters of a generative model based on the recognized emotion, and a means for the terminal to order food from an external organization based on a recipe, thereby enabling the generation and suggestion of optimal recipes tailored to the user's emotion, and further enabling the user to quickly and conveniently order food based on the recipe.

[0355] "User" refers to an individual who uses the system to input ingredients, make recipe suggestions, and place food orders.

[0356] "Ingredients" are food items owned by the user that are the basis for recipes.

[0357] "Terminal" refers to a device operated by a user, such as a smartphone, tablet, or PC.

[0358] A "server" is a remote computer that receives and processes data sent from a terminal.

[0359] A "generative model" refers to an algorithm that analyzes input data and generates a recipe.

[0360] A "recipe" refers to the cooking steps and methods generated by the system based on ingredients.

[0361] "Emotion recognition" refers to technology that analyzes and recognizes a user's emotions based on sensor data from cameras, microphones, etc.

[0362] "Parameter adjustment" refers to changing the settings of a generative model based on user emotions and other data.

[0363] "External organization" refers to a restaurant or grocery store that is external to the system and provides the food ordered by the user.

[0364] The system of the present invention is a system that generates new recipes based on ingredients that the user has, and then allows the user to order food based on the recipes. Specific embodiments for carrying out the invention are described below.

[0365] The system consists of the following components:

[0366] Terminal: A device operated by a user, such as a smartphone, tablet, or PC. The terminal is used to input ingredients, display recipes, and order food through an application.

[0367] Server: A remote computer that receives user-entered data and performs emotion recognition and recipe generation. Here, we can use Python and Flask as the backend server.

[0368] Generative model: An algorithm that analyzes input data and generates recipes. Recipes are suggested using generative AI models.

[0369] Emotion recognition engine: A technology that analyzes user emotions based on sensor data from cameras, microphones, etc. It uses libraries such as EmotionRecognizer.

[0370] External Organization: An organization outside the system that provides the food that users order, such as a restaurant or grocery store.

[0371] Example of a system

[0372] Step 1: Enter ingredients

[0373] The user launches the application and displays the ingredient input screen. The user enters the ingredients they have on hand and taps the Add button to add them to the list. After entering all the ingredients, they tap the Send button, and the ingredient list is sent from the device to the server in JSON format.

[0374] Step 2: Emotion Recognition

[0375] The server receives the ingredient list entered by the user and simultaneously acquires sensor data from the camera and microphone. The emotion recognition engine analyzes this sensor data and recognizes the user's emotions.

[0376] Step 3: Recipe generation

[0377] The server adjusts the parameters of the generative AI model based on the emotional data obtained by the emotion recognition engine. For example, if the user feels like relaxing, the server adjusts the generative model to prioritize ingredients and cooking methods that have a relaxing effect. The server then inputs the ingredient list into the generative AI model and generates a new recipe.

[0378] Step 4: View recipes and order

[0379] The generated recipe information is converted back to JSON format and sent from the server to the device. The device parses the received recipe information, displays it to the user, and provides the user with the option to order food from an external organization (e.g., a nearby restaurant) based on this recipe.

[0380] Examples of prompt statements

[0381] By inputting prompts to the generative AI model as follows, a recipe based on the user's emotions is generated:

[0382] The user wants to relax. Generate a relaxing recipe using the following ingredients:

[0383] chicken meat

[0384] Carrots

[0385] potatoes

[0386] The system embodying this invention allows users to receive optimal recipe suggestions based on their emotions and quickly order food based on those recipes, thereby improving the user experience and dramatically increasing convenience.

[0387] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0388] Step 1:

[0389] The user launches the application. The user accesses the ingredient input screen and inputs the ingredients they have on hand. After inputting each ingredient, they tap the Add button to add it to the list. After inputting all ingredients, they tap the Send button. This saves the input ingredient list to the device. The input data is the name of the specific ingredients the user has in their hand.

[0390] Step 2:

[0391] The device sends the ingredient list to the server. The device converts the saved ingredient list into JSON format, stores it in the HTTP request body, and sends it to the server. The sent data is the ingredient list in JSON format. The server receives this request.

[0392] Step 3:

[0393] The server parses the received ingredient list. The server extracts the JSON data from the HTTP request body and defines the ingredient list as a data structure. This data is the parsed ingredient list.

[0394] Step 4:

[0395] The server recognizes the user's emotions. The server activates an emotion recognition engine using sensor data acquired from the device, such as a camera or microphone. The sensor data includes facial expressions and tone of voice, and is analyzed using a library such as EmotionRecognizer. The result of this analysis is the recognized user's emotion data.

[0396] Step 5:

[0397] The server adjusts the parameters of the generative model based on the emotion recognition results. The server adaptively changes the parameters of the generative AI model based on the recognized emotion data. For example, when a user is feeling stressed, the server sets the parameters to prioritize recipes that have a relaxing effect. These adjusted parameters are used as input for the generative model.

[0398] Step 6:

[0399] The server generates a recipe using the generative AI model. The server inputs the adjusted parameters and ingredient list into the generative AI model to generate a new recipe. The output data is the generated recipe information.

[0400] Step 7:

[0401] The server converts the generated recipe information into JSON format and sends it to the terminal as an HTTP response. The response data contains detailed information about the generated recipe. The terminal receives this response.

[0402] Step 8:

[0403] The device analyzes the received recipe information and displays it to the user. The generated recipe is displayed on the recipe browsing screen. The device also provides an option to order food from an external organization based on the recipe. For example, it displays a link or button to order directly from a restaurant or grocery store. This allows the user to quickly order food based on the suggested recipe.

[0404] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0405] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0406] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0407] [Second embodiment]

[0408] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0409] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0410] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0411] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0412] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0413] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0414] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0415] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0416] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0417] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0418] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0419] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0420] This invention relates to a system that generates and provides new recipes using ingredients that a user has at home when the user inputs them into an app. Specifically, the system includes a means for the user to input ingredients, a means for the device to send the input ingredient list to a server, a means for the server to generate a recipe using a generative AI model, a means for sending the generated recipe to the device, and a means for the device to display the generated recipe to the user.

[0421] System program processing overview

[0422] The system is programmed as follows:

[0423] Ingredients input stage

[0424] The user launches the app and displays the ingredient input screen. They enter the ingredients they want to use in the text box and tap the "Add" button to add them to the ingredient list. After entering all the ingredients, they tap the "Submit" button.

[0425] Ingredient list sending stage

[0426] The device sends the ingredient list entered by the user to the server. The device converts the ingredient list into JSON format, stores it in the HTTP request body, and sends it to the server.

[0427] Recipe generation stage by the server

[0428] The server receives and analyzes the ingredient list. The server passes the ingredient list as input to the generative AI model to generate a recipe. The generated recipe is formatted in JSON format and sent to the device as an HTTP response.

[0429] Recipe display stage

[0430] The terminal analyzes the response received from the server and acquires the recipe information. The terminal displays the recipe screen and provides the generated recipe to the user.

[0431] Specific examples

[0432] For example, if a user has chicken, carrots, and potatoes at home, a new recipe will be generated using those ingredients.

[0433] The user inputs ingredients into the app, sequentially entering "chicken," "carrots," and "potatoes." When the user taps the "Send" button, the device sends the list of ingredients to the server.

[0434] The server analyzes the received ingredient list and inputs it into a generative AI model. The generative AI model generates a recipe such as "chicken and vegetable stew," and the server formats the recipe information and sends it to the device.

[0435] The device receives this recipe information and shows the recipe display screen to the user, who can then start cooking based on the recipe for "chicken and vegetable stew."

[0436] In this way, this system helps users efficiently use ingredients on hand and enjoy new dishes. At the same time, it reduces food waste and increases the variety of daily dishes. It is also possible to customize recipes generated according to the user's preferences and generate recipes specialized for health management.

[0437] The processing flow will be explained below.

[0438] Step 1:

[0439] The user launches the app and the app's home screen appears.

[0440] Step 2:

[0441] The user taps the "Create a new recipe" button. The ingredient input screen appears.

[0442] Step 3:

[0443] The user enters the ingredients they want to use in the text box, entering "chicken," "carrots," and "potatoes" in that order, and taps the "Add" button to add them to the ingredients list.

[0444] Step 4:

[0445] The user enters all ingredients and taps the "Submit" button. The device retrieves the ingredient list and proceeds to the next step.

[0446] Step 5:

[0447] The device converts the ingredient list into JSON format and stores the JSON data of the ingredient list in the HTTP request body.

[0448] Step 6:

[0449] The device sends an HTTP request containing JSON data to the server. The request URL is used to send the data to the server.

[0450] Step 7:

[0451] The server receives the HTTP request and extracts the JSON data of the ingredient list from the request body.

[0452] Step 8:

[0453] The server parses the JSON data of the ingredient list and obtains information about each ingredient: "chicken," "carrot," and "potato."

[0454] Step 9:

[0455] The server passes the list of ingredients as input to the generative AI model, which then generates a recipe based on the ingredients.

[0456] Step 10:

[0457] The server converts the generated recipe information into JSON format, generates an HTTP response, and stores the JSON recipe information in the response body.

[0458] Step 11:

[0459] The server sends an HTTP response including the generated recipe information to the terminal.

[0460] Step 12:

[0461] The device receives the HTTP response and obtains the recipe information JSON data from the response body.

[0462] Step 13:

[0463] The device parses the JSON data and extracts the recipe information for "Chicken and Vegetable Stew." It then generates the recipe screen.

[0464] Step 14:

[0465] The device displays a recipe viewing screen, providing the user with information such as the recipe title, ingredients, and cooking steps.

[0466] Step 15:

[0467] The user checks the displayed recipes and starts cooking based on the recipe "Chicken and Vegetable Stew."

[0468] These are the specific processing steps of this system, which allows users to efficiently use ingredients at home and obtain new cooking recipes.

[0469] Example 1

[0470] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0471] In today's busy lifestyles, users need a way to efficiently utilize ingredients on hand and easily create new recipes. However, conventional systems have complicated processes for inputting ingredients and creating recipes, which is time-consuming for users. In addition, they lack the functionality to create recipes specialized for health management or customized recipes based on user preferences.

[0472] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0473] In this invention, the server includes a means for analyzing the ingredient list and converting it into a prompt statement, a means for inputting the prompt statement into a generative AI model to generate a recipe, and a means for formatting the generated recipe into JSON format and sending it to the terminal as an HTTP response. This allows the user to easily input ingredients on hand and automatically generate a new recipe using the generative AI model. It also simultaneously enables parameter adjustment according to the user's preferences and the generation of recipes specialized for health management.

[0474] 1. A "user" is an individual who uses the system to input ingredients on hand and generate recipes.

[0475] 2. "Terminal" means the collection of hardware and software used by a user to input ingredients, transmit the input to the server, and receive and display the generated recipe.

[0476] 3. "Server" means a collection of hardware and software that analyzes the ingredient list received from the Terminal, generates a recipe using a generative AI model, and sends it to the Terminal.

[0477] 4. An "ingredient list" is a collection of data that a user enters to identify the ingredients they wish to use.

[0478] 5. "JSON format" stands for JavaScript Object Notation and is a standard for representing data in a lightweight text format.

[0479] 6. "HTTP request" is a part of the communication protocol used by a device to send data to a server.

[0480] 7. "HTTP response" is part of the communication protocol used by the server to send data to the terminal.

[0481] 8. A "prompt" is a sentence that is input to the generative AI model to generate a recipe based on a list of ingredients.

[0482] 9. “Generative AI Model” means an artificial intelligence model for generating new recipes based on input prompts.

[0483] 10. "Recipe display screen" is an interface that allows the terminal to visually display to the user recipe information received from the server.

[0484] The above definitions of each word concretely show the functions and roles of the system.

[0485] MODE FOR CARRYING OUT THE INVENTION

[0486] An embodiment of this invention is a system that generates and provides new recipes using ingredients when a user inputs ingredients available at home into an application. Specifically, the system includes a means for the user to input ingredients, a means for the terminal to send the input ingredient list to a server, a means for the server to generate a recipe using a generative AI model, a means for sending the generated recipe to the terminal, and a means for the terminal to display the generated recipe to the user.

[0487] Hardware and software used

[0488] Hardware

[0489] Devices such as smartphones, tablets, and computers

[0490] Server for recipe generation and data processing

[0491] software

[0492] Application for inputting ingredients

[0493] Data communication using the HTTP protocol

[0494] Generative AI models (e.g., OpenAI GPT-3)

[0495] System configuration and operation

[0496] Below, each component of this system and its operation will be explained in detail.

[0497] The user launches the application on their smartphone or tablet and displays the ingredient input screen. The user enters the ingredients they want to use one by one into the text box and taps the "Add" button to add them to the ingredient list. After entering all the ingredients, the user taps the "Submit" button.

[0498] The device obtains the ingredient list entered by the user and converts it to JSON format. The converted JSON data is stored in the body of the HTTP request and an HTTP POST request is sent to the specified endpoint on the server. This request includes the ingredient list entered by the user.

[0499] The server receives the HTTP request and extracts the ingredient list from the request body. It then parses the ingredient list to generate a prompt. For example, it generates a prompt like, "The user has the following ingredients: - chicken - carrots - potatoes. Please generate a new recipe using these ingredients."

[0500] The generated prompt is input into a generative AI model to generate a recipe. The generative AI model generates a new recipe (e.g., "chicken and vegetable stew") and formats the result in JSON format. The formatted JSON data is stored in the body of an HTTP response and sent to the device.

[0501] The device receives the HTTP response from the server and analyzes the JSON data in the response. Based on the analysis results, it creates a recipe display screen and displays the new recipe to the user. The user can then start cooking according to the recipe.

[0502] Specific examples

[0503] For example, if a user has chicken, carrots, and potatoes at home, a new recipe will be generated using those ingredients.

[0504] The user launches the application, enters "chicken," "carrots," and "potatoes," and taps the "Send" button. The device sends the ingredient list to the server, which analyzes the received ingredient list and inputs it into the generative AI model. The generative AI model generates a recipe (e.g., "chicken and vegetable stew"), and the server formats the recipe information in JSON format and sends it to the device. The device receives this recipe information and displays the recipe display screen to the user. The user can then start cooking based on the "chicken and vegetable stew" recipe.

[0505] In this way, users can efficiently use ingredients they have on hand and enjoy new dishes. In addition, by adjusting the parameters of the generative AI model, it is possible to customize recipes according to user preferences and generate recipes specialized for health management.

[0506] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0507] The flow of this system's program processing

[0508] Step 1:

[0509] The user launches the application and the ingredient input screen is displayed. The user enters the ingredients they want to use one by one into the text box and taps the "Add" button. For example, the user can enter "chicken," "carrot," and "potato" in that order. When the user taps the "Submit" button, all the ingredients entered are included in the list.

[0510] Input: User input of ingredients (e.g. "chicken", "carrot", "potato")

[0511] Output: List of ingredients (e.g. ["chicken", "carrot", "potato"])

[0512] Step 2:

[0513] The device converts the ingredient list entered by the user into JSON format and stores it in the body of the HTTP request. This request is then sent to the server. Specifically, the ingredient list is sent to the specified endpoint on the server with the "application / json" content type.

[0514] Input: List of ingredients (e.g. ["chicken", "carrot", "potato"])

[0515] Output: JSON format data and HTTP request (e.g., {"ingredients":["chicken", "carrot", "potato"]})

[0516] Step 3:

[0517] The server analyzes the received HTTP request and retrieves the list of ingredients from the request body. It then uses the retrieved ingredient list to generate a prompt. This prompt is input into the generative AI model and serves as an instruction for generating a recipe. For example, the prompt might be in the form, "The user has the following ingredients: - chicken - carrots - potatoes. Please generate a new recipe using these ingredients."

[0518] Input: HTTP request, JSON format data (e.g., {"ingredients":["chicken", "carrot", "potato"]})

[0519] Output: Prompt text (e.g., "The user has the following ingredients: - chicken - carrots - potatoes. Generate a new recipe using these ingredients.")

[0520] Step 4:

[0521] The server inputs the prompt into a generative AI model to generate a new recipe. A generative AI model (e.g., OpenAI GPT-3) is used to generate a recipe based on the prompt. The generated recipe is then converted back to JSON format and stored in the body of the HTTP response.

[0522] Input: Prompt text (e.g., "The user has the following ingredients: - chicken - carrots - potatoes. Generate a new recipe using these ingredients.")

[0523] Output: Recipe information (e.g., "Chicken and vegetable stew"), JSON-formatted recipe data (e.g., {"Recipe": "Chicken and vegetable stew", "Steps": ["Cut the chicken", "Chop the vegetables", "Simmer"]})

[0524] Step 5:

[0525] The server stores the JSON data of the generated recipe in the HTTP response body and sends it to the terminal.

[0526] Input: JSON-formatted recipe data (e.g., {"Recipe": "Chicken and vegetable stew", "Steps": ["Cut the chicken", "Chop the vegetables", "Simmer"]})

[0527] Output: HTTP response, recipe data in JSON format

[0528] Step 6:

[0529] The device receives the HTTP response from the server and retrieves the JSON-formatted recipe data from the response body. It then analyzes this data to create a recipe display screen and presents it to the user. The user can then check the recipe and start cooking.

[0530] Input: HTTP response, JSON-formatted recipe data (e.g., {"Recipe": "Chicken and vegetable stew", "Steps": ["Cut the chicken", "Chop the vegetables", "Simmer"]})

[0531] Output: Recipe display screen, visual display of recipe information

[0532] The above is the flow of processing in the program of this system and the specific operations performed at each step.

[0533] (Application example 1)

[0534] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0535] While existing systems exist that efficiently utilize ingredients available at home to generate new recipes, they do not include a means for easily obtaining ingredients or seasonings that are in short supply, which means that users have to go shopping separately to obtain ingredients that are not available at home, resulting in inconvenience. This also leads to food waste and limits the variety of dishes that can be made. Therefore, there is a need for the development of a new system that allows users to easily obtain ingredients and seasonings that are in short supply.

[0536] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0537] In this invention, the server includes a means for inputting ingredients, a means for transmitting an ingredient list to the server, a means for generating a recipe using a generative model, a means for transmitting the generated recipe to a terminal, a means for displaying the generated recipe, and a means for the user to order delivery of ingredients and seasonings that the user is running low on. This allows the user to make the most of ingredients they have on hand and efficiently obtain ingredients and seasonings that they are running low on, enabling them to enjoy new variations of dishes.

[0538] "User" refers to an individual who uses this system to input ingredients, create recipes, and place delivery orders.

[0539] "Terminal" refers to an electronic device used by a user, such as a smartphone or computer.

[0540] The "ingredient list" refers to a list of multiple ingredients entered by the user.

[0541] "Server" refers to a computer system that runs on the cloud or a network and generates recipes and communicates data.

[0542] A "generative model" refers to a machine learning algorithm or AI system that creates new recipes based on input data.

[0543] A "recipe" refers to detailed information about cooking steps and ingredients generated using an ingredient list.

[0544] "Delivery order" refers to the act of a user ordering ingredients or seasonings they are running low on via the Internet and having them delivered to their home.

[0545] "Display means" refers to a function for visually showing the recipe generated on the terminal to the user.

[0546] The present invention provides a system that allows a user to make the most of ingredients on hand and efficiently obtain ingredients and seasonings that are in short supply. Specific embodiments of the present invention are described below.

[0547] System Configuration

[0548] The system includes a terminal used by a user (e.g., a smartphone or tablet), a server, and a network that communicates via the Internet.

[0549] Hardware and software used

[0550] Device: Smartphone or tablet (device where user inputs ingredients and displays recipe and delivery order information).

[0551] Server: A computer system in the cloud (implemented using Python and Flask).

[0552] Generative AI models: Generative models such as OpenAI GPT-4.

[0553] Network: A network for transmitting data between devices and servers over the Internet.

[0554] Function and process description

[0555] Enter ingredients

[0556] The user launches the app on their device and inputs the ingredients they have on hand. On the ingredient input screen, the user enters the ingredients they want to use in the text box and taps the "Add" button to add them to the ingredient list.

[0557] Send ingredient list

[0558] The terminal converts the ingredient list entered by the user into JSON format, stores it in the HTTP request body, and sends it to the server.

[0559] Recipe Generation

[0560] The server receives the input ingredient list and analyzes it. The analyzed ingredient list is input into the generative AI model, and a new recipe is generated. The recipe generated by the generative AI model is formatted in JSON and sent to the device as an HTTP response.

[0561] View recipes and order delivery

[0562] The device analyzes the response received from the server and obtains the generated recipe information. The user can then cook based on this information. They can also order delivery of missing ingredients and seasonings based on the generated recipe.

[0563] Specific examples

[0564] For example, if a user has "tomatoes," "olive oil," and "basil" at home, the following processing will occur:

[0565] Example prompt sentence:

[0566] "Suggest a new recipe using the following ingredients: tomatoes, olive oil, and basil."

[0567] Generated recipe example:

[0568] Tomato and Basil Pasta: Finely chop the tomatoes and fry them in olive oil. Add the basil and fry some more, then mix with the cooked pasta. Season with salt and pepper and top with Parmesan cheese.

[0569] The user can start cooking based on this generated recipe information, and can easily order delivery to get any ingredients they are missing.

[0570] This allows users to make the most of the ingredients they have on hand, expanding the variety of dishes they can make, while also reducing food waste.

[0571] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0572] Step 1:

[0573] Enter ingredients

[0574] The user launches the app on their device and displays the ingredient input screen. The user types the ingredients they have at home into the text box and taps the "Add" button. This adds the ingredients to the list. After entering all the ingredients, the user taps the "Submit" button.

[0575] Input: Ingredients entered by the user.

[0576] Output: A list of ingredients is generated.

[0577] Step 2:

[0578] Send ingredient list

[0579] The device converts the ingredient list entered by the user into JSON format, then sends this JSON data to the server as an HTTP POST request.

[0580] Input: A list of ingredients entered by the user.

[0581] Output: A JSON formatted list of ingredients is sent to the server.

[0582] Step 3:

[0583] Recipe Generation

[0584] The server parses the JSON-formatted ingredient list received from the device. The server calls the API of a generative AI model (e.g., OpenAI GPT-4) and provides the ingredient list as input data. The generative AI model generates a new recipe based on the input data. The generated recipe is formatted in JSON format.

[0585] Input: A list of ingredients in JSON format.

[0586] Output: The new recipe generated by the generative AI model.

[0587] Step 4:

[0588] Send recipe

[0589] The server sends the created recipe to the terminal as an HTTP response in JSON format.

[0590] Input: The generated recipe.

[0591] Output: The recipe in JSON format is sent to the terminal.

[0592] Step 5:

[0593] Recipe display

[0594] The device analyzes the JSON response received from the server. The device displays the analyzed recipe information on the recipe screen. The user can check the displayed recipe and start cooking. It is also possible to order delivery of missing ingredients and seasonings based on the recipe.

[0595] Input: Recipe information in JSON format.

[0596] Output: The parsed recipe information is displayed to the user.

[0597] Through these steps, users can not only efficiently utilize the ingredients they have on hand and create new recipes, but also easily order delivery of ingredients or seasonings they are running low on.

[0598] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0599] This invention relates to a system that generates and provides new recipes using ingredients when a user inputs ingredients available at home into an application. Specifically, the system includes a means for the user to input ingredients, a means for the terminal to send the input ingredient list to a server, a means for the server to generate a recipe using a generative model, a means for the terminal to send the generated recipe, a means for the terminal to display the generated recipe to the user, and an emotion engine that recognizes the user's emotions and adjusts the parameters of a recipe generation model based on the emotions.

[0600] System program processing overview

[0601] The system is programmed as follows:

[0602] Ingredients input stage

[0603] The user launches the app and the ingredient input screen is displayed. The user enters the ingredients they want to use and taps the Add button to add them to the ingredient list. After entering all ingredients, they tap the Send button.

[0604] Ingredient list sending stage

[0605] The device sends the ingredient list entered by the user to the server. The device converts the ingredient list into JSON format, stores it in the HTTP request body, and sends it to the server.

[0606] Emotion recognition stage by server

[0607] The server receives the HTTP request and extracts the JSON data of the ingredient list from the request body. Next, the emotion engine recognizes the user's emotion. Emotions are acquired and analyzed using sensor data from cameras, microphones, etc.

[0608] Recipe Generation Stage

[0609] The server analyzes the ingredient list and adjusts the parameters of the generative AI model based on the user's perceived emotions. For example, if the user is feeling stressed, the model is adjusted to prioritize ingredients and cooking methods that have a relaxing effect. The server then inputs the ingredient list into the generative AI model to generate a recipe.

[0610] Recipe submission stage

[0611] The server converts the generated recipe information into JSON format and sends it to the terminal as an HTTP response.

[0612] Recipe display stage

[0613] The terminal analyzes the response received from the server and acquires the recipe information. The terminal displays a recipe viewing screen and provides the created recipe to the user.

[0614] Specific examples

[0615] For example, if a user wants to get new recipes using "chicken," "carrots," and "potatoes," he or she will use this system.

[0616] The user inputs ingredients into the app, entering "chicken," "carrots," and "potatoes" respectively. When the user taps the send button, the device sends the list of ingredients to the server.

[0617] The server analyzes the received ingredient list and also recognizes the user's emotions from camera and microphone data. For example, if the server recognizes that the user is tired, the generative AI model generates a recipe with an emphasis on "dishes that have a refreshing effect." The generated recipe is "chicken and carrot soup."

[0618] The server converts this recipe information into JSON format and sends it to the device.

[0619] The device receives the recipe information and shows the recipe display screen to the user, who then begins cooking based on the recipe for "chicken and carrot soup."

[0620] In this way, the system can generate optimal recipes based on the user's emotions, improving the user's cooking experience by providing new dishes without wasting ingredients on hand.

[0621] The processing flow will be explained below.

[0622] Step 1:

[0623] The user launches the app and the home screen appears.

[0624] Step 2:

[0625] The user taps the "Create a new recipe" button. The ingredient input screen appears.

[0626] Step 3:

[0627] The user enters the ingredients they want to use in the text box, entering "chicken," "carrots," and "potatoes" in that order, and taps the "Add" button to add them to the ingredients list.

[0628] Step 4:

[0629] The user enters all ingredients and taps the "Submit" button. The device retrieves the ingredient list and proceeds to the next step.

[0630] Step 5:

[0631] The device converts the ingredient list into JSON format and stores the JSON data of the ingredient list in the HTTP request body.

[0632] Step 6:

[0633] The device sends an HTTP request containing JSON data to the server. The request URL is used to send the data to the server.

[0634] Step 7:

[0635] The server receives the HTTP request and extracts the JSON data of the ingredient list from the request body.

[0636] Step 8:

[0637] The server parses the JSON data of the ingredient list and obtains information about each ingredient: "chicken," "carrot," and "potato."

[0638] Step 9:

[0639] The server activates the emotion engine to recognize the user's emotions. It acquires sensor data from cameras and microphones and analyzes the user's facial expressions and tone of voice.

[0640] Step 10:

[0641] The server determines the user's emotion from the analysis results of the emotion engine. For example, it determines that the user is tired.

[0642] Step 11:

[0643] The server adjusts the parameters of the generative AI model based on the analysis results of the emotion engine, for example, by changing the settings to prioritize dishes with a refreshing effect.

[0644] Step 12:

[0645] The server passes the list of ingredients as input to the generative AI model and generates a recipe, for example, "chicken and carrot soup."

[0646] Step 13:

[0647] The server converts the generated recipe information into JSON format, generates an HTTP response, and stores the JSON recipe information in the response body.

[0648] Step 14:

[0649] The server sends an HTTP response including the generated recipe information to the terminal.

[0650] Step 15:

[0651] The device receives the HTTP response and obtains the recipe information JSON data from the response body.

[0652] Step 16:

[0653] The device parses the JSON data and extracts the recipe information for "Chicken and Carrot Soup." It then generates the recipe screen.

[0654] Step 17:

[0655] The device displays a recipe viewing screen, providing the user with information such as the recipe title, ingredients, and cooking steps.

[0656] Step 18:

[0657] The user checks the displayed recipes and starts cooking based on the "Chicken and Carrot Soup" recipe.

[0658] These are the specific processing steps in this system, which allows users to obtain the best recipes based on the ingredients and emotions at home.

[0659] Example 2

[0660] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0661] Conventional recipe generation systems are unable to generate recipes that reflect the user's current emotions and preferences, and the recipes they provide may not meet the user's expectations or needs. Furthermore, they have issues with being unable to efficiently utilize the ingredients the user has on hand, and not proposing recipes that are appropriate from a health management perspective.

[0662] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for recognizing a user's emotion using an emotion recognition engine, a means for adjusting parameters of a generative AI model based on the recognized emotion and generating a recipe, and a means for transmitting the generated recipe to a terminal. This makes it possible to provide an optimal recipe according to the user's emotion and preferences.

[0663] "User" refers to a person who uses the system to input ingredients and obtain recipes.

[0664] A "terminal" is a device that allows a user to input ingredients, transmits an ingredient list to a server, and displays the recipe received from the server.

[0665] The "server" is a computer system that receives the ingredient list sent by the user, recognizes the user's emotions using an emotion recognition engine, generates a recipe using a generative AI model, and sends the generated recipe to the terminal.

[0666] An "emotion recognition engine" is a system or algorithm that analyzes data obtained from the user's camera or microphone to recognize the user's emotions.

[0667] A "generative AI model" is a machine learning model that automatically generates new recipes based on input data, and is particularly capable of adjusting parameters according to the user's emotions.

[0668] The "ingredient list" is a list of multiple ingredients that the user wants to use, and is data in a format that is sent from the terminal to the server.

[0669] A "recipe" is information that provides detailed instructions for making a dish using ingredients, and is generated by a generative AI model.

[0670] "Parameter adjustment" refers to changing the internal settings of a generative AI model to change its behavior depending on specific conditions or situations.

[0671] This invention relates to a system that generates and provides new recipes using ingredients that a user has at home when they input them into an application. This system is composed of the following specific hardware and software.

[0672] Hardware and software used

[0673] The terminal used by the user is a mobile device such as a smartphone or tablet, and an application for inputting ingredients is installed on this terminal.

[0674] The server is a typical server providing cloud computing services, and includes web server software (e.g., Apache or Nginx) to process HTTP requests, software to run a generative AI model (e.g., GPT-4) that analyzes ingredient lists and generates recipes, and image and voice analysis technologies required to run an emotion recognition engine.

[0675] Examples of user actions

[0676] When a user launches the app, an ingredient input screen appears. This screen has a text box and an "Add" button and a "Submit" button. The user inputs ingredients such as "chicken," "carrots," and "potatoes," and taps the "Add" button to add them to the ingredient list. Once all ingredients have been input, the user taps the "Submit" button to confirm the list.

[0677] Specific examples of terminal operation

[0678] The device sends the ingredient list entered by the user to the server. At this time, the ingredient list is converted to JSON format and sent to the server as the body of the HTTP request.

[0679] Example of server operation

[0680] When the server receives an HTTP request, it extracts the JSON data from the request body, parses the list of ingredients, and uses an emotion recognition engine to recognize the user's emotion. For example, it can recognize the user's emotion of "tired" through sensor data collected from the camera and microphone.

[0681] Next, the server adjusts the parameters of the generative AI model based on the recognized user's emotions and generates a recipe. For example, a prompt commanding the model to generate a "refreshing recipe for a tired user" is input. Suppose the generated recipe is "chicken and carrot soup."

[0682] "Please suggest a refreshing dish using chicken, carrots, and potatoes for tired users."

[0683] Use a prompt such as:

[0684] The server converts the generated recipe information into JSON format and sends it to the terminal as an HTTP response. The terminal analyzes the received response and obtains the generated recipe information (e.g., "Chicken and Carrot Soup"). Finally, the terminal displays the recipe viewing screen to the user.

[0685] In this way, the system can provide optimal recipes based on the user's emotions and preferences, providing a new cooking experience without wasting the ingredients the user has.

[0686] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0687] Step 1:

[0688] The user launches the app and displays the ingredient input screen.

[0689] Specific operation: The user taps the app icon on their smartphone or tablet to launch the application. After launching, the ingredient input screen is displayed.

[0690] Input and output: The input is a user operation (tapping the app icon), and the output is the display of the ingredient input screen.

[0691] Step 2:

[0692] The user enters the ingredients they want to use and taps the add button.

[0693] Specific behavior: The user enters "chicken" in the text box and taps the "Add" button. This displays the ingredients in the text box as a list. This operation is repeated for "carrots" and "potatoes."

[0694] Input and output: The input is the ingredient name and tapping the add button, and the output is the display of the ingredient list.

[0695] Step 3:

[0696] After the user enters all the ingredients, he taps the submit button.

[0697] Specific actions: Once the user has entered all the required ingredients, they tap the "Submit" button on the screen.

[0698] Input and Output: The input is the tap of the send button, and the output is the trigger of the send request.

[0699] Step 4:

[0700] The terminal transmits the ingredient list input by the user to the server.

[0701] Specific operation: The device serializes the input ingredient list into JSON format, stores it in the HTTP request body, and sends it to the server.

[0702] Input and Output: The input is the list of ingredients entered by the user, and the output is the HTTP request sent to the server.

[0703] Step 5:

[0704] The server receives the HTTP request and extracts the JSON data of the ingredient list from the request body.

[0705] Specific behavior: The server parses the received HTTP request and extracts a JSON-formatted list of ingredients from the request body.

[0706] Input and Output: The input is the HTTP request sent from the terminal, and the output is the parsed JSON data of the ingredient list.

[0707] Step 6:

[0708] The server uses an emotion recognition engine to recognize the user's emotion.

[0709] Specific operation: The server analyzes sensor data obtained from the camera and microphone and recognizes the user's emotions. For example, it detects emotions such as "tired" from the user's facial expression and tone of voice.

[0710] Input and Output: The input is the sensor data and the ingredient list, and the output is the recognized user emotion data.

[0711] Step 7:

[0712] The server adjusts the parameters of the generative AI model based on the recognized emotions and generates a recipe.

[0713] Specific operation: The server generates a prompt sentence based on the emotion recognition result and inputs it into a generative AI model (e.g., GPT-4). Let's assume that the generated recipe is "chicken and carrot soup."

[0714] Input and Output: The input is the recognized emotion data and the ingredients list, and the output is the generated recipe.

[0715] Step 8:

[0716] The server converts the generated recipe information into JSON format and sends it to the terminal as an HTTP response.

[0717] Specific operation: The server serializes the generated recipe information in JSON format, stores it in the body of the HTTP response, and sends it to the terminal.

[0718] Input and Output: The input is the generated recipe information, and the output is the HTTP response sent to the terminal.

[0719] Step 9:

[0720] The terminal analyzes the response received from the server and acquires the recipe information.

[0721] Specific operation: The device parses the HTTP response received and extracts the recipe information.

[0722] Input and Output: The input is the HTTP response received from the server, and the output is the extracted recipe information.

[0723] Step 10:

[0724] The terminal displays a recipe viewing screen and provides the generated recipe to the user.

[0725] Specific operation: The device displays a recipe viewing screen and provides the user with detailed information such as ingredients, steps, and tips for the dish.

[0726] Input and Output: The input is the extracted recipe information and the output is the recipe screen that is displayed to the user.

[0727] This system allows users to efficiently use ingredients they have on hand and obtain the optimal recipe based on their emotional state.

[0728] (Application example 2)

[0729] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0730] Conventional recipe generation systems only generate recipes based on the ingredients a user has on hand, and do not consider the user's emotions or mood when suggesting recipes. Furthermore, they lack a function for instantly ordering food based on the generated recipe, forcing users to search for purchasing locations and ordering methods themselves based on the suggested recipe. This results in a poor user experience and a lack of convenience. To solve these problems, a system is needed that generates recipes that consider the user's emotions and seamlessly processes food orders from external organizations.

[0731] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0732] In this invention, the server includes a means for recognizing a user's emotion, a means for adjusting parameters of a generative model based on the recognized emotion, and a means for the terminal to order food from an external organization based on a recipe, thereby enabling the generation and suggestion of optimal recipes tailored to the user's emotion, and further enabling the user to quickly and conveniently order food based on the recipe.

[0733] "User" refers to an individual who uses the system to input ingredients, make recipe suggestions, and place food orders.

[0734] "Ingredients" are food items owned by the user that are the basis for recipes.

[0735] "Terminal" refers to a device operated by a user, such as a smartphone, tablet, or PC.

[0736] A "server" is a remote computer that receives and processes data sent from a terminal.

[0737] A "generative model" refers to an algorithm that analyzes input data and generates a recipe.

[0738] A "recipe" refers to the cooking steps and methods generated by the system based on ingredients.

[0739] "Emotion recognition" refers to technology that analyzes and recognizes a user's emotions based on sensor data from cameras, microphones, etc.

[0740] "Parameter adjustment" refers to changing the settings of a generative model based on user emotions and other data.

[0741] "External organization" refers to a restaurant or grocery store that is external to the system and provides the food ordered by the user.

[0742] The system of the present invention is a system that generates new recipes based on ingredients that the user has, and then allows the user to order food based on the recipes. Specific embodiments for carrying out the invention are described below.

[0743] The system consists of the following components:

[0744] Terminal: A device operated by a user, such as a smartphone, tablet, or PC. The terminal is used to input ingredients, display recipes, and order food through an application.

[0745] Server: A remote computer that receives user-entered data and performs emotion recognition and recipe generation. Here, we can use Python and Flask as the backend server.

[0746] Generative model: An algorithm that analyzes input data and generates recipes. Recipes are suggested using generative AI models.

[0747] Emotion recognition engine: A technology that analyzes user emotions based on sensor data from cameras, microphones, etc. It uses libraries such as EmotionRecognizer.

[0748] External Organization: An organization outside the system that provides the food that users order, such as a restaurant or grocery store.

[0749] Example of a system

[0750] Step 1: Enter ingredients

[0751] The user launches the application and displays the ingredient input screen. The user enters the ingredients they have on hand and taps the Add button to add them to the list. After entering all the ingredients, they tap the Send button, and the ingredient list is sent from the device to the server in JSON format.

[0752] Step 2: Emotion Recognition

[0753] The server receives the ingredient list entered by the user and simultaneously acquires sensor data from the camera and microphone. The emotion recognition engine analyzes this sensor data and recognizes the user's emotions.

[0754] Step 3: Recipe generation

[0755] The server adjusts the parameters of the generative AI model based on the emotional data obtained by the emotion recognition engine. For example, if the user feels like relaxing, the server adjusts the generative model to prioritize ingredients and cooking methods that have a relaxing effect. The server then inputs the ingredient list into the generative AI model and generates a new recipe.

[0756] Step 4: View recipes and order

[0757] The generated recipe information is converted back to JSON format and sent from the server to the device. The device parses the received recipe information, displays it to the user, and provides the user with the option to order food from an external organization (e.g., a nearby restaurant) based on this recipe.

[0758] Examples of prompt statements

[0759] By inputting prompts to the generative AI model as follows, a recipe based on the user's emotions is generated:

[0760] The user wants to relax. Generate a relaxing recipe using the following ingredients:

[0761] chicken meat

[0762] Carrots

[0763] potatoes

[0764] The system embodying this invention allows users to receive optimal recipe suggestions based on their emotions and quickly order food based on those recipes, thereby improving the user experience and dramatically increasing convenience.

[0765] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0766] Step 1:

[0767] The user launches the application. The user accesses the ingredient input screen and inputs the ingredients they have on hand. After inputting each ingredient, they tap the Add button to add it to the list. After inputting all ingredients, they tap the Send button. This saves the input ingredient list to the device. The input data is the name of the specific ingredients the user has in their hand.

[0768] Step 2:

[0769] The device sends the ingredient list to the server. The device converts the saved ingredient list into JSON format, stores it in the HTTP request body, and sends it to the server. The sent data is the ingredient list in JSON format. The server receives this request.

[0770] Step 3:

[0771] The server parses the received ingredient list. The server extracts the JSON data from the HTTP request body and defines the ingredient list as a data structure. This data is the parsed ingredient list.

[0772] Step 4:

[0773] The server recognizes the user's emotions. The server activates an emotion recognition engine using sensor data acquired from the device, such as a camera or microphone. The sensor data includes facial expressions and tone of voice, and is analyzed using a library such as EmotionRecognizer. The result of this analysis is the recognized user's emotion data.

[0774] Step 5:

[0775] The server adjusts the parameters of the generative model based on the emotion recognition results. The server adaptively changes the parameters of the generative AI model based on the recognized emotion data. For example, when a user is feeling stressed, the server sets the parameters to prioritize recipes that have a relaxing effect. These adjusted parameters are used as input for the generative model.

[0776] Step 6:

[0777] The server generates a recipe using the generative AI model. The server inputs the adjusted parameters and ingredient list into the generative AI model to generate a new recipe. The output data is the generated recipe information.

[0778] Step 7:

[0779] The server converts the generated recipe information into JSON format and sends it to the terminal as an HTTP response. The response data contains detailed information about the generated recipe. The terminal receives this response.

[0780] Step 8:

[0781] The device analyzes the received recipe information and displays it to the user. The generated recipe is displayed on the recipe browsing screen. The device also provides an option to order food from an external organization based on the recipe. For example, it displays a link or button to order directly from a restaurant or grocery store. This allows the user to quickly order food based on the suggested recipe.

[0782] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0783] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0784] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0785] [Third embodiment]

[0786] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0787] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0788] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0789] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0790] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0791] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0792] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0793] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0794] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0795] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0796] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0797] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0798] This invention relates to a system that generates and provides new recipes using ingredients that a user has at home when the user inputs them into an app. Specifically, the system includes a means for the user to input ingredients, a means for the device to send the input ingredient list to a server, a means for the server to generate a recipe using a generative AI model, a means for sending the generated recipe to the device, and a means for the device to display the generated recipe to the user.

[0799] System program processing overview

[0800] The system is programmed as follows:

[0801] Ingredients input stage

[0802] The user launches the app and displays the ingredient input screen. They enter the ingredients they want to use in the text box and tap the "Add" button to add them to the ingredient list. After entering all the ingredients, they tap the "Submit" button.

[0803] Ingredient list sending stage

[0804] The device sends the ingredient list entered by the user to the server. The device converts the ingredient list into JSON format, stores it in the HTTP request body, and sends it to the server.

[0805] Recipe generation stage by the server

[0806] The server receives and analyzes the ingredient list. The server passes the ingredient list as input to the generative AI model to generate a recipe. The generated recipe is formatted in JSON format and sent to the device as an HTTP response.

[0807] Recipe display stage

[0808] The terminal analyzes the response received from the server and acquires the recipe information. The terminal displays the recipe screen and provides the generated recipe to the user.

[0809] Specific examples

[0810] For example, if a user has chicken, carrots, and potatoes at home, a new recipe will be generated using those ingredients.

[0811] The user inputs ingredients into the app, sequentially entering "chicken," "carrots," and "potatoes." When the user taps the "Send" button, the device sends the list of ingredients to the server.

[0812] The server analyzes the received ingredient list and inputs it into a generative AI model. The generative AI model generates a recipe such as "chicken and vegetable stew," and the server formats the recipe information and sends it to the device.

[0813] The device receives this recipe information and shows the recipe display screen to the user, who can then start cooking based on the recipe for "chicken and vegetable stew."

[0814] In this way, this system helps users efficiently use ingredients on hand and enjoy new dishes. At the same time, it reduces food waste and increases the variety of daily dishes. It is also possible to customize recipes generated according to the user's preferences and generate recipes specialized for health management.

[0815] The processing flow will be explained below.

[0816] Step 1:

[0817] The user launches the app and the app's home screen appears.

[0818] Step 2:

[0819] The user taps the "Create a new recipe" button. The ingredient input screen appears.

[0820] Step 3:

[0821] The user enters the ingredients they want to use in the text box, entering "chicken," "carrots," and "potatoes" in that order, and taps the "Add" button to add them to the ingredients list.

[0822] Step 4:

[0823] The user enters all ingredients and taps the "Submit" button. The device retrieves the ingredient list and proceeds to the next step.

[0824] Step 5:

[0825] The device converts the ingredient list into JSON format and stores the JSON data of the ingredient list in the HTTP request body.

[0826] Step 6:

[0827] The device sends an HTTP request containing JSON data to the server. The request URL is used to send the data to the server.

[0828] Step 7:

[0829] The server receives the HTTP request and extracts the JSON data of the ingredient list from the request body.

[0830] Step 8:

[0831] The server parses the JSON data of the ingredient list and obtains information about each ingredient: "chicken," "carrot," and "potato."

[0832] Step 9:

[0833] The server passes the list of ingredients as input to the generative AI model, which then generates a recipe based on the ingredients.

[0834] Step 10:

[0835] The server converts the generated recipe information into JSON format, generates an HTTP response, and stores the JSON recipe information in the response body.

[0836] Step 11:

[0837] The server sends an HTTP response including the generated recipe information to the terminal.

[0838] Step 12:

[0839] The device receives the HTTP response and obtains the recipe information JSON data from the response body.

[0840] Step 13:

[0841] The device parses the JSON data and extracts the recipe information for "Chicken and Vegetable Stew." It then generates the recipe screen.

[0842] Step 14:

[0843] The device displays a recipe viewing screen, providing the user with information such as the recipe title, ingredients, and cooking steps.

[0844] Step 15:

[0845] The user checks the displayed recipes and starts cooking based on the recipe "Chicken and Vegetable Stew."

[0846] These are the specific processing steps of this system, which allows users to efficiently use ingredients at home and obtain new cooking recipes.

[0847] Example 1

[0848] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0849] In today's busy lifestyles, users need a way to efficiently utilize ingredients on hand and easily create new recipes. However, conventional systems have complicated processes for inputting ingredients and creating recipes, which is time-consuming for users. In addition, they lack the functionality to create recipes specialized for health management or customized recipes based on user preferences.

[0850] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0851] In this invention, the server includes a means for analyzing the ingredient list and converting it into a prompt statement, a means for inputting the prompt statement into a generative AI model to generate a recipe, and a means for formatting the generated recipe into JSON format and sending it to the terminal as an HTTP response. This allows the user to easily input ingredients on hand and automatically generate a new recipe using the generative AI model. It also simultaneously enables parameter adjustment according to the user's preferences and the generation of recipes specialized for health management.

[0852] 1. A "user" is an individual who uses the system to input ingredients on hand and generate recipes.

[0853] 2. "Terminal" means the collection of hardware and software used by a user to input ingredients, transmit the input to the server, and receive and display the generated recipe.

[0854] 3. "Server" means a collection of hardware and software that analyzes the ingredient list received from the Terminal, generates a recipe using a generative AI model, and sends it to the Terminal.

[0855] 4. An "ingredient list" is a collection of data that a user enters to identify the ingredients they wish to use.

[0856] 5. "JSON format" stands for JavaScript Object Notation and is a standard for representing data in a lightweight text format.

[0857] 6. "HTTP request" is a part of the communication protocol used by a device to send data to a server.

[0858] 7. "HTTP response" is part of the communication protocol used by the server to send data to the terminal.

[0859] 8. A "prompt" is a sentence that is input to the generative AI model to generate a recipe based on a list of ingredients.

[0860] 9. “Generative AI Model” means an artificial intelligence model for generating new recipes based on input prompts.

[0861] 10. "Recipe display screen" is an interface that allows the terminal to visually display to the user recipe information received from the server.

[0862] The above definitions of each word concretely show the functions and roles of the system.

[0863] MODE FOR CARRYING OUT THE INVENTION

[0864] An embodiment of this invention is a system that generates and provides new recipes using ingredients when a user inputs ingredients available at home into an application. Specifically, the system includes a means for the user to input ingredients, a means for the terminal to send the input ingredient list to a server, a means for the server to generate a recipe using a generative AI model, a means for sending the generated recipe to the terminal, and a means for the terminal to display the generated recipe to the user.

[0865] Hardware and software used

[0866] Hardware

[0867] Devices such as smartphones, tablets, and computers

[0868] Server for recipe generation and data processing

[0869] software

[0870] Application for inputting ingredients

[0871] Data communication using the HTTP protocol

[0872] Generative AI models (e.g., OpenAI GPT-3)

[0873] System configuration and operation

[0874] Below, each component of this system and its operation will be explained in detail.

[0875] The user launches the application on their smartphone or tablet and displays the ingredient input screen. The user enters the ingredients they want to use one by one into the text box and taps the "Add" button to add them to the ingredient list. After entering all the ingredients, the user taps the "Submit" button.

[0876] The device obtains the ingredient list entered by the user and converts it to JSON format. The converted JSON data is stored in the body of the HTTP request and an HTTP POST request is sent to the specified endpoint on the server. This request includes the ingredient list entered by the user.

[0877] The server receives the HTTP request and extracts the ingredient list from the request body. It then parses the ingredient list to generate a prompt. For example, it generates a prompt like, "The user has the following ingredients: - chicken - carrots - potatoes. Please generate a new recipe using these ingredients."

[0878] The generated prompt is input into a generative AI model to generate a recipe. The generative AI model generates a new recipe (e.g., "chicken and vegetable stew") and formats the result in JSON format. The formatted JSON data is stored in the body of an HTTP response and sent to the device.

[0879] The device receives the HTTP response from the server and analyzes the JSON data in the response. Based on the analysis results, it creates a recipe display screen and displays the new recipe to the user. The user can then start cooking according to the recipe.

[0880] Specific examples

[0881] For example, if a user has chicken, carrots, and potatoes at home, a new recipe will be generated using those ingredients.

[0882] The user launches the application, enters "chicken," "carrots," and "potatoes," and taps the "Send" button. The device sends the ingredient list to the server, which analyzes the received ingredient list and inputs it into the generative AI model. The generative AI model generates a recipe (e.g., "chicken and vegetable stew"), and the server formats the recipe information in JSON format and sends it to the device. The device receives this recipe information and displays the recipe display screen to the user. The user can then start cooking based on the "chicken and vegetable stew" recipe.

[0883] In this way, users can efficiently use ingredients they have on hand and enjoy new dishes. In addition, by adjusting the parameters of the generative AI model, it is possible to customize recipes according to user preferences and generate recipes specialized for health management.

[0884] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0885] The flow of this system's program processing

[0886] Step 1:

[0887] The user launches the application and the ingredient input screen is displayed. The user enters the ingredients they want to use one by one into the text box and taps the "Add" button. For example, the user can enter "chicken," "carrot," and "potato" in that order. When the user taps the "Submit" button, all the ingredients entered are included in the list.

[0888] Input: User input of ingredients (e.g. "chicken", "carrot", "potato")

[0889] Output: List of ingredients (e.g. ["chicken", "carrot", "potato"])

[0890] Step 2:

[0891] The device converts the ingredient list entered by the user into JSON format and stores it in the body of the HTTP request. This request is then sent to the server. Specifically, the ingredient list is sent to the specified endpoint on the server with the "application / json" content type.

[0892] Input: List of ingredients (e.g. ["chicken", "carrot", "potato"])

[0893] Output: JSON format data and HTTP request (e.g., {"ingredients":["chicken", "carrot", "potato"]})

[0894] Step 3:

[0895] The server analyzes the received HTTP request and retrieves the list of ingredients from the request body. It then uses the retrieved ingredient list to generate a prompt. This prompt is input into the generative AI model and serves as an instruction for generating a recipe. For example, the prompt might be in the form, "The user has the following ingredients: - chicken - carrots - potatoes. Please generate a new recipe using these ingredients."

[0896] Input: HTTP request, JSON format data (e.g., {"ingredients":["chicken", "carrot", "potato"]})

[0897] Output: Prompt text (e.g., "The user has the following ingredients: - chicken - carrots - potatoes. Generate a new recipe using these ingredients.")

[0898] Step 4:

[0899] The server inputs the prompt into a generative AI model to generate a new recipe. A generative AI model (e.g., OpenAI GPT-3) is used to generate a recipe based on the prompt. The generated recipe is then converted back to JSON format and stored in the body of the HTTP response.

[0900] Input: Prompt text (e.g., "The user has the following ingredients: - chicken - carrots - potatoes. Generate a new recipe using these ingredients.")

[0901] Output: Recipe information (e.g., "Chicken and vegetable stew"), JSON-formatted recipe data (e.g., {"Recipe": "Chicken and vegetable stew", "Steps": ["Cut the chicken", "Chop the vegetables", "Simmer"]})

[0902] Step 5:

[0903] The server stores the JSON data of the generated recipe in the HTTP response body and sends it to the terminal.

[0904] Input: JSON-formatted recipe data (e.g., {"Recipe": "Chicken and vegetable stew", "Steps": ["Cut the chicken", "Chop the vegetables", "Simmer"]})

[0905] Output: HTTP response, recipe data in JSON format

[0906] Step 6:

[0907] The device receives the HTTP response from the server and retrieves the JSON-formatted recipe data from the response body. It then analyzes this data to create a recipe display screen and presents it to the user. The user can then check the recipe and start cooking.

[0908] Input: HTTP response, JSON-formatted recipe data (e.g., {"Recipe": "Chicken and vegetable stew", "Steps": ["Cut the chicken", "Chop the vegetables", "Simmer"]})

[0909] Output: Recipe display screen, visual display of recipe information

[0910] The above is the flow of processing in the program of this system and the specific operations performed at each step.

[0911] (Application example 1)

[0912] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0913] While existing systems exist that efficiently utilize ingredients available at home to generate new recipes, they do not include a means for easily obtaining ingredients or seasonings that are in short supply, which means that users have to go shopping separately to obtain ingredients that are not available at home, resulting in inconvenience. This also leads to food waste and limits the variety of dishes that can be made. Therefore, there is a need for the development of a new system that allows users to easily obtain ingredients and seasonings that are in short supply.

[0914] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0915] In this invention, the server includes a means for inputting ingredients, a means for transmitting an ingredient list to the server, a means for generating a recipe using a generative model, a means for transmitting the generated recipe to a terminal, a means for displaying the generated recipe, and a means for the user to order delivery of ingredients and seasonings that the user is running low on. This allows the user to make the most of ingredients they have on hand and efficiently obtain ingredients and seasonings that they are running low on, enabling them to enjoy new variations of dishes.

[0916] "User" refers to an individual who uses this system to input ingredients, create recipes, and place delivery orders.

[0917] "Terminal" refers to an electronic device used by a user, such as a smartphone or computer.

[0918] The "ingredient list" refers to a list of multiple ingredients entered by the user.

[0919] "Server" refers to a computer system that runs on the cloud or a network and generates recipes and communicates data.

[0920] A "generative model" refers to a machine learning algorithm or AI system that creates new recipes based on input data.

[0921] A "recipe" refers to detailed information about cooking steps and ingredients generated using an ingredient list.

[0922] "Delivery order" refers to the act of a user ordering ingredients or seasonings they are running low on via the Internet and having them delivered to their home.

[0923] "Display means" refers to a function for visually showing the recipe generated on the terminal to the user.

[0924] The present invention provides a system that allows a user to make the most of ingredients on hand and efficiently obtain ingredients and seasonings that are in short supply. Specific embodiments of the present invention are described below.

[0925] System Configuration

[0926] The system includes a terminal used by a user (e.g., a smartphone or tablet), a server, and a network that communicates via the Internet.

[0927] Hardware and software used

[0928] Device: Smartphone or tablet (device where user inputs ingredients and displays recipe and delivery order information).

[0929] Server: A computer system in the cloud (implemented using Python and Flask).

[0930] Generative AI models: Generative models such as OpenAI GPT-4.

[0931] Network: A network for transmitting data between devices and servers over the Internet.

[0932] Function and process description

[0933] Enter ingredients

[0934] The user launches the app on their device and inputs the ingredients they have on hand. On the ingredient input screen, the user enters the ingredients they want to use in the text box and taps the "Add" button to add them to the ingredient list.

[0935] Send ingredient list

[0936] The terminal converts the ingredient list entered by the user into JSON format, stores it in the HTTP request body, and sends it to the server.

[0937] Recipe Generation

[0938] The server receives the input ingredient list and analyzes it. The analyzed ingredient list is input into the generative AI model, and a new recipe is generated. The recipe generated by the generative AI model is formatted in JSON and sent to the device as an HTTP response.

[0939] View recipes and order delivery

[0940] The device analyzes the response received from the server and obtains the generated recipe information. The user can then cook based on this information. They can also order delivery of missing ingredients and seasonings based on the generated recipe.

[0941] Specific examples

[0942] For example, if a user has "tomatoes," "olive oil," and "basil" at home, the following processing will occur:

[0943] Example prompt sentence:

[0944] "Suggest a new recipe using the following ingredients: tomatoes, olive oil, and basil."

[0945] Generated recipe example:

[0946] Tomato and Basil Pasta: Finely chop the tomatoes and fry them in olive oil. Add the basil and fry some more, then mix with the cooked pasta. Season with salt and pepper and top with Parmesan cheese.

[0947] The user can start cooking based on this generated recipe information, and can easily order delivery to get any ingredients they are missing.

[0948] This allows users to make the most of the ingredients they have on hand, expanding the variety of dishes they can make, while also reducing food waste.

[0949] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0950] Step 1:

[0951] Enter ingredients

[0952] The user launches the app on their device and displays the ingredient input screen. The user types the ingredients they have at home into the text box and taps the "Add" button. This adds the ingredients to the list. After entering all the ingredients, the user taps the "Submit" button.

[0953] Input: Ingredients entered by the user.

[0954] Output: A list of ingredients is generated.

[0955] Step 2:

[0956] Send ingredient list

[0957] The device converts the ingredient list entered by the user into JSON format, then sends this JSON data to the server as an HTTP POST request.

[0958] Input: A list of ingredients entered by the user.

[0959] Output: A JSON formatted list of ingredients is sent to the server.

[0960] Step 3:

[0961] Recipe Generation

[0962] The server parses the JSON-formatted ingredient list received from the device. The server calls the API of a generative AI model (e.g., OpenAI GPT-4) and provides the ingredient list as input data. The generative AI model generates a new recipe based on the input data. The generated recipe is formatted in JSON format.

[0963] Input: A list of ingredients in JSON format.

[0964] Output: The new recipe generated by the generative AI model.

[0965] Step 4:

[0966] Send recipe

[0967] The server sends the created recipe to the terminal as an HTTP response in JSON format.

[0968] Input: The generated recipe.

[0969] Output: The recipe in JSON format is sent to the terminal.

[0970] Step 5:

[0971] Recipe display

[0972] The device analyzes the JSON response received from the server. The device displays the analyzed recipe information on the recipe screen. The user can check the displayed recipe and start cooking. It is also possible to order delivery of missing ingredients and seasonings based on the recipe.

[0973] Input: Recipe information in JSON format.

[0974] Output: The parsed recipe information is displayed to the user.

[0975] Through these steps, users can not only efficiently utilize the ingredients they have on hand and create new recipes, but also easily order delivery of ingredients or seasonings they are running low on.

[0976] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0977] This invention relates to a system that generates and provides new recipes using ingredients when a user inputs ingredients available at home into an application. Specifically, the system includes a means for the user to input ingredients, a means for the terminal to send the input ingredient list to a server, a means for the server to generate a recipe using a generative model, a means for the terminal to send the generated recipe, a means for the terminal to display the generated recipe to the user, and an emotion engine that recognizes the user's emotions and adjusts the parameters of a recipe generation model based on the emotions.

[0978] System program processing overview

[0979] The system is programmed as follows:

[0980] Ingredients input stage

[0981] The user launches the app and the ingredient input screen is displayed. The user enters the ingredients they want to use and taps the Add button to add them to the ingredient list. After entering all ingredients, they tap the Send button.

[0982] Ingredient list sending stage

[0983] The device sends the ingredient list entered by the user to the server. The device converts the ingredient list into JSON format, stores it in the HTTP request body, and sends it to the server.

[0984] Emotion recognition stage by server

[0985] The server receives the HTTP request and extracts the JSON data of the ingredient list from the request body. Next, the emotion engine recognizes the user's emotion. Emotions are acquired and analyzed using sensor data from cameras, microphones, etc.

[0986] Recipe Generation Stage

[0987] The server analyzes the ingredient list and adjusts the parameters of the generative AI model based on the user's perceived emotions. For example, if the user is feeling stressed, the model is adjusted to prioritize ingredients and cooking methods that have a relaxing effect. The server then inputs the ingredient list into the generative AI model to generate a recipe.

[0988] Recipe submission stage

[0989] The server converts the generated recipe information into JSON format and sends it to the terminal as an HTTP response.

[0990] Recipe display stage

[0991] The terminal analyzes the response received from the server and acquires the recipe information. The terminal displays a recipe viewing screen and provides the created recipe to the user.

[0992] Specific examples

[0993] For example, if a user wants to get new recipes using "chicken," "carrots," and "potatoes," he or she will use this system.

[0994] The user inputs ingredients into the app, entering "chicken," "carrots," and "potatoes" respectively. When the user taps the send button, the device sends the list of ingredients to the server.

[0995] The server analyzes the received ingredient list and also recognizes the user's emotions from camera and microphone data. For example, if the server recognizes that the user is tired, the generative AI model generates a recipe with an emphasis on "dishes that have a refreshing effect." The generated recipe is "chicken and carrot soup."

[0996] The server converts this recipe information into JSON format and sends it to the device.

[0997] The device receives the recipe information and shows the recipe display screen to the user, who then begins cooking based on the recipe for "chicken and carrot soup."

[0998] In this way, the system can generate optimal recipes based on the user's emotions, improving the user's cooking experience by providing new dishes without wasting ingredients on hand.

[0999] The processing flow will be explained below.

[1000] Step 1:

[1001] The user launches the app and the home screen appears.

[1002] Step 2:

[1003] The user taps the "Create a new recipe" button. The ingredient input screen appears.

[1004] Step 3:

[1005] The user enters the ingredients they want to use in the text box, entering "chicken," "carrots," and "potatoes" in that order, and taps the "Add" button to add them to the ingredients list.

[1006] Step 4:

[1007] The user enters all ingredients and taps the "Submit" button. The device retrieves the ingredient list and proceeds to the next step.

[1008] Step 5:

[1009] The device converts the ingredient list into JSON format and stores the JSON data of the ingredient list in the HTTP request body.

[1010] Step 6:

[1011] The device sends an HTTP request containing JSON data to the server. The request URL is used to send the data to the server.

[1012] Step 7:

[1013] The server receives the HTTP request and extracts the JSON data of the ingredient list from the request body.

[1014] Step 8:

[1015] The server parses the JSON data of the ingredient list and obtains information about each ingredient: "chicken," "carrot," and "potato."

[1016] Step 9:

[1017] The server activates the emotion engine to recognize the user's emotions. It acquires sensor data from cameras and microphones and analyzes the user's facial expressions and tone of voice.

[1018] Step 10:

[1019] The server determines the user's emotion from the analysis results of the emotion engine. For example, it determines that the user is tired.

[1020] Step 11:

[1021] The server adjusts the parameters of the generative AI model based on the analysis results of the emotion engine, for example, by changing the settings to prioritize dishes with a refreshing effect.

[1022] Step 12:

[1023] The server passes the list of ingredients as input to the generative AI model and generates a recipe, for example, "chicken and carrot soup."

[1024] Step 13:

[1025] The server converts the generated recipe information into JSON format, generates an HTTP response, and stores the JSON recipe information in the response body.

[1026] Step 14:

[1027] The server sends an HTTP response including the generated recipe information to the terminal.

[1028] Step 15:

[1029] The device receives the HTTP response and obtains the recipe information JSON data from the response body.

[1030] Step 16:

[1031] The device parses the JSON data and extracts the recipe information for "Chicken and Carrot Soup." It then generates the recipe screen.

[1032] Step 17:

[1033] The device displays a recipe viewing screen, providing the user with information such as the recipe title, ingredients, and cooking steps.

[1034] Step 18:

[1035] The user checks the displayed recipes and starts cooking based on the "Chicken and Carrot Soup" recipe.

[1036] These are the specific processing steps in this system, which allows users to obtain the best recipes based on the ingredients and emotions at home.

[1037] Example 2

[1038] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1039] Conventional recipe generation systems are unable to generate recipes that reflect the user's current emotions and preferences, and the recipes they provide may not meet the user's expectations or needs. Furthermore, they have issues with being unable to efficiently utilize the ingredients the user has on hand, and not proposing recipes that are appropriate from a health management perspective.

[1040] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for recognizing a user's emotion using an emotion recognition engine, a means for adjusting parameters of a generative AI model based on the recognized emotion and generating a recipe, and a means for transmitting the generated recipe to a terminal. This makes it possible to provide an optimal recipe according to the user's emotion and preferences.

[1041] "User" refers to a person who uses the system to input ingredients and obtain recipes.

[1042] A "terminal" is a device that allows a user to input ingredients, transmits an ingredient list to a server, and displays the recipe received from the server.

[1043] The "server" is a computer system that receives the ingredient list sent by the user, recognizes the user's emotions using an emotion recognition engine, generates a recipe using a generative AI model, and sends the generated recipe to the terminal.

[1044] An "emotion recognition engine" is a system or algorithm that analyzes data obtained from the user's camera or microphone to recognize the user's emotions.

[1045] A "generative AI model" is a machine learning model that automatically generates new recipes based on input data, and is particularly capable of adjusting parameters according to the user's emotions.

[1046] The "ingredient list" is a list of multiple ingredients that the user wants to use, and is data in a format that is sent from the terminal to the server.

[1047] A "recipe" is information that provides detailed instructions for making a dish using ingredients, and is generated by a generative AI model.

[1048] "Parameter adjustment" refers to changing the internal settings of a generative AI model to change its behavior depending on specific conditions or situations.

[1049] This invention relates to a system that generates and provides new recipes using ingredients that a user has at home when they input them into an application. This system is composed of the following specific hardware and software.

[1050] Hardware and software used

[1051] The terminal used by the user is a mobile device such as a smartphone or tablet, and an application for inputting ingredients is installed on this terminal.

[1052] The server is a typical server providing cloud computing services, and includes web server software (e.g., Apache or Nginx) to process HTTP requests, software to run a generative AI model (e.g., GPT-4) that analyzes ingredient lists and generates recipes, and image and voice analysis technologies required to run an emotion recognition engine.

[1053] Examples of user actions

[1054] When a user launches the app, an ingredient input screen appears. This screen has a text box and an "Add" button and a "Submit" button. The user inputs ingredients such as "chicken," "carrots," and "potatoes," and taps the "Add" button to add them to the ingredient list. Once all ingredients have been input, the user taps the "Submit" button to confirm the list.

[1055] Specific examples of terminal operation

[1056] The device sends the ingredient list entered by the user to the server. At this time, the ingredient list is converted to JSON format and sent to the server as the body of the HTTP request.

[1057] Example of server operation

[1058] When the server receives an HTTP request, it extracts the JSON data from the request body, parses the list of ingredients, and uses an emotion recognition engine to recognize the user's emotion. For example, it can recognize the user's emotion of "tired" through sensor data collected from the camera and microphone.

[1059] Next, the server adjusts the parameters of the generative AI model based on the recognized user's emotions and generates a recipe. For example, a prompt commanding the model to generate a "refreshing recipe for a tired user" is input. Suppose the generated recipe is "chicken and carrot soup."

[1060] "Please suggest a refreshing dish using chicken, carrots, and potatoes for tired users."

[1061] Use a prompt such as:

[1062] The server converts the generated recipe information into JSON format and sends it to the terminal as an HTTP response. The terminal analyzes the received response and obtains the generated recipe information (e.g., "Chicken and Carrot Soup"). Finally, the terminal displays the recipe viewing screen to the user.

[1063] In this way, the system can provide optimal recipes based on the user's emotions and preferences, providing a new cooking experience without wasting the ingredients the user has.

[1064] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1065] Step 1:

[1066] The user launches the app and displays the ingredient input screen.

[1067] Specific operation: The user taps the app icon on their smartphone or tablet to launch the application. After launching, the ingredient input screen is displayed.

[1068] Input and output: The input is a user operation (tapping the app icon), and the output is the display of the ingredient input screen.

[1069] Step 2:

[1070] The user enters the ingredients they want to use and taps the add button.

[1071] Specific behavior: The user enters "chicken" in the text box and taps the "Add" button. This displays the ingredients in the text box as a list. This operation is repeated for "carrots" and "potatoes."

[1072] Input and output: The input is the ingredient name and tapping the add button, and the output is the display of the ingredient list.

[1073] Step 3:

[1074] After the user enters all the ingredients, he taps the submit button.

[1075] Specific actions: Once the user has entered all the required ingredients, they tap the "Submit" button on the screen.

[1076] Input and Output: The input is the tap of the send button, and the output is the trigger of the send request.

[1077] Step 4:

[1078] The terminal transmits the ingredient list input by the user to the server.

[1079] Specific operation: The device serializes the input ingredient list into JSON format, stores it in the HTTP request body, and sends it to the server.

[1080] Input and Output: The input is the list of ingredients entered by the user, and the output is the HTTP request sent to the server.

[1081] Step 5:

[1082] The server receives the HTTP request and extracts the JSON data of the ingredient list from the request body.

[1083] Specific behavior: The server parses the received HTTP request and extracts a JSON-formatted list of ingredients from the request body.

[1084] Input and Output: The input is the HTTP request sent from the terminal, and the output is the parsed JSON data of the ingredient list.

[1085] Step 6:

[1086] The server uses an emotion recognition engine to recognize the user's emotion.

[1087] Specific operation: The server analyzes sensor data obtained from the camera and microphone and recognizes the user's emotions. For example, it detects emotions such as "tired" from the user's facial expression and tone of voice.

[1088] Input and Output: The input is the sensor data and the ingredient list, and the output is the recognized user emotion data.

[1089] Step 7:

[1090] The server adjusts the parameters of the generative AI model based on the recognized emotions and generates a recipe.

[1091] Specific operation: The server generates a prompt sentence based on the emotion recognition result and inputs it into a generative AI model (e.g., GPT-4). Let's assume that the generated recipe is "chicken and carrot soup."

[1092] Input and Output: The input is the recognized emotion data and the ingredients list, and the output is the generated recipe.

[1093] Step 8:

[1094] The server converts the generated recipe information into JSON format and sends it to the terminal as an HTTP response.

[1095] Specific operation: The server serializes the generated recipe information in JSON format, stores it in the body of the HTTP response, and sends it to the terminal.

[1096] Input and Output: The input is the generated recipe information, and the output is the HTTP response sent to the terminal.

[1097] Step 9:

[1098] The terminal analyzes the response received from the server and acquires the recipe information.

[1099] Specific operation: The device parses the HTTP response received and extracts the recipe information.

[1100] Input and Output: The input is the HTTP response received from the server, and the output is the extracted recipe information.

[1101] Step 10:

[1102] The terminal displays a recipe viewing screen and provides the generated recipe to the user.

[1103] Specific operation: The device displays a recipe viewing screen and provides the user with detailed information such as ingredients, steps, and tips for the dish.

[1104] Input and Output: The input is the extracted recipe information and the output is the recipe screen that is displayed to the user.

[1105] This system allows users to efficiently use ingredients they have on hand and obtain the optimal recipe based on their emotional state.

[1106] (Application example 2)

[1107] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1108] Conventional recipe generation systems only generate recipes based on the ingredients a user has on hand, and do not consider the user's emotions or mood when suggesting recipes. Furthermore, they lack a function for instantly ordering food based on the generated recipe, forcing users to search for purchasing locations and ordering methods themselves based on the suggested recipe. This results in a poor user experience and a lack of convenience. To solve these problems, a system is needed that generates recipes that consider the user's emotions and seamlessly processes food orders from external organizations.

[1109] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1110] In this invention, the server includes a means for recognizing a user's emotion, a means for adjusting parameters of a generative model based on the recognized emotion, and a means for the terminal to order food from an external organization based on a recipe, thereby enabling the generation and suggestion of optimal recipes tailored to the user's emotion, and further enabling the user to quickly and conveniently order food based on the recipe.

[1111] "User" refers to an individual who uses the system to input ingredients, make recipe suggestions, and place food orders.

[1112] "Ingredients" are food items owned by the user that are the basis for recipes.

[1113] "Terminal" refers to a device operated by a user, such as a smartphone, tablet, or PC.

[1114] A "server" is a remote computer that receives and processes data sent from a terminal.

[1115] A "generative model" refers to an algorithm that analyzes input data and generates a recipe.

[1116] A "recipe" refers to the cooking steps and methods generated by the system based on ingredients.

[1117] "Emotion recognition" refers to technology that analyzes and recognizes a user's emotions based on sensor data from cameras, microphones, etc.

[1118] "Parameter adjustment" refers to changing the settings of a generative model based on user emotions and other data.

[1119] "External organization" refers to a restaurant or grocery store that is external to the system and provides the food ordered by the user.

[1120] The system of the present invention is a system that generates new recipes based on ingredients that the user has, and then allows the user to order food based on the recipes. Specific embodiments for carrying out the invention are described below.

[1121] The system consists of the following components:

[1122] Terminal: A device operated by a user, such as a smartphone, tablet, or PC. The terminal is used to input ingredients, display recipes, and order food through an application.

[1123] Server: A remote computer that receives user-entered data and performs emotion recognition and recipe generation. Here, we can use Python and Flask as the backend server.

[1124] Generative model: An algorithm that analyzes input data and generates recipes. Recipes are suggested using generative AI models.

[1125] Emotion recognition engine: A technology that analyzes user emotions based on sensor data from cameras, microphones, etc. It uses libraries such as EmotionRecognizer.

[1126] External Organization: An organization outside the system that provides the food that users order, such as a restaurant or grocery store.

[1127] Example of a system

[1128] Step 1: Enter ingredients

[1129] The user launches the application and displays the ingredient input screen. The user enters the ingredients they have on hand and taps the Add button to add them to the list. After entering all the ingredients, they tap the Send button, and the ingredient list is sent from the device to the server in JSON format.

[1130] Step 2: Emotion Recognition

[1131] The server receives the ingredient list entered by the user and simultaneously acquires sensor data from the camera and microphone. The emotion recognition engine analyzes this sensor data and recognizes the user's emotions.

[1132] Step 3: Recipe generation

[1133] The server adjusts the parameters of the generative AI model based on the emotional data obtained by the emotion recognition engine. For example, if the user feels like relaxing, the server adjusts the generative model to prioritize ingredients and cooking methods that have a relaxing effect. The server then inputs the ingredient list into the generative AI model and generates a new recipe.

[1134] Step 4: View recipes and order

[1135] The generated recipe information is converted back to JSON format and sent from the server to the device. The device parses the received recipe information, displays it to the user, and provides the user with the option to order food from an external organization (e.g., a nearby restaurant) based on this recipe.

[1136] Examples of prompt statements

[1137] By inputting prompts to the generative AI model as follows, a recipe based on the user's emotions is generated:

[1138] The user wants to relax. Generate a relaxing recipe using the following ingredients:

[1139] chicken meat

[1140] Carrots

[1141] potatoes

[1142] The system embodying this invention allows users to receive optimal recipe suggestions based on their emotions and quickly order food based on those recipes, thereby improving the user experience and dramatically increasing convenience.

[1143] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1144] Step 1:

[1145] The user launches the application. The user accesses the ingredient input screen and inputs the ingredients they have on hand. After inputting each ingredient, they tap the Add button to add it to the list. After inputting all ingredients, they tap the Send button. This saves the input ingredient list to the device. The input data is the name of the specific ingredients the user has in their hand.

[1146] Step 2:

[1147] The device sends the ingredient list to the server. The device converts the saved ingredient list into JSON format, stores it in the HTTP request body, and sends it to the server. The sent data is the ingredient list in JSON format. The server receives this request.

[1148] Step 3:

[1149] The server parses the received ingredient list. The server extracts the JSON data from the HTTP request body and defines the ingredient list as a data structure. This data is the parsed ingredient list.

[1150] Step 4:

[1151] The server recognizes the user's emotions. The server activates an emotion recognition engine using sensor data acquired from the device, such as a camera or microphone. The sensor data includes facial expressions and tone of voice, and is analyzed using a library such as EmotionRecognizer. The result of this analysis is the recognized user's emotion data.

[1152] Step 5:

[1153] The server adjusts the parameters of the generative model based on the emotion recognition results. The server adaptively changes the parameters of the generative AI model based on the recognized emotion data. For example, when a user is feeling stressed, the server sets the parameters to prioritize recipes that have a relaxing effect. These adjusted parameters are used as input for the generative model.

[1154] Step 6:

[1155] The server generates a recipe using the generative AI model. The server inputs the adjusted parameters and ingredient list into the generative AI model to generate a new recipe. The output data is the generated recipe information.

[1156] Step 7:

[1157] The server converts the generated recipe information into JSON format and sends it to the terminal as an HTTP response. The response data contains detailed information about the generated recipe. The terminal receives this response.

[1158] Step 8:

[1159] The device analyzes the received recipe information and displays it to the user. The generated recipe is displayed on the recipe browsing screen. The device also provides an option to order food from an external organization based on the recipe. For example, it displays a link or button to order directly from a restaurant or grocery store. This allows the user to quickly order food based on the suggested recipe.

[1160] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1161] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1162] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1163] [Fourth embodiment]

[1164] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1165] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1166] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1167] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1168] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1169] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1170] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1171] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1172] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1173] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1174] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1175] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1176] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1177] This invention relates to a system that generates and provides new recipes using ingredients that a user has at home when the user inputs them into an app. Specifically, the system includes a means for the user to input ingredients, a means for the device to send the input ingredient list to a server, a means for the server to generate a recipe using a generative AI model, a means for sending the generated recipe to the device, and a means for the device to display the generated recipe to the user.

[1178] System program processing overview

[1179] The system is programmed as follows:

[1180] Ingredients input stage

[1181] The user launches the app and displays the ingredient input screen. They enter the ingredients they want to use in the text box and tap the "Add" button to add them to the ingredient list. After entering all the ingredients, they tap the "Submit" button.

[1182] Ingredient list sending stage

[1183] The device sends the ingredient list entered by the user to the server. The device converts the ingredient list into JSON format, stores it in the HTTP request body, and sends it to the server.

[1184] Recipe generation stage by the server

[1185] The server receives and analyzes the ingredient list. The server passes the ingredient list as input to the generative AI model to generate a recipe. The generated recipe is formatted in JSON format and sent to the device as an HTTP response.

[1186] Recipe display stage

[1187] The terminal analyzes the response received from the server and acquires the recipe information. The terminal displays the recipe screen and provides the generated recipe to the user.

[1188] Specific examples

[1189] For example, if a user has chicken, carrots, and potatoes at home, a new recipe will be generated using those ingredients.

[1190] The user inputs ingredients into the app, sequentially entering "chicken," "carrots," and "potatoes." When the user taps the "Send" button, the device sends the list of ingredients to the server.

[1191] The server analyzes the received ingredient list and inputs it into a generative AI model. The generative AI model generates a recipe such as "chicken and vegetable stew," and the server formats the recipe information and sends it to the device.

[1192] The device receives this recipe information and shows the recipe display screen to the user, who can then start cooking based on the recipe for "chicken and vegetable stew."

[1193] In this way, this system helps users efficiently use ingredients on hand and enjoy new dishes. At the same time, it reduces food waste and increases the variety of daily dishes. It is also possible to customize recipes generated according to the user's preferences and generate recipes specialized for health management.

[1194] The processing flow will be explained below.

[1195] Step 1:

[1196] The user launches the app and the app's home screen appears.

[1197] Step 2:

[1198] The user taps the "Create a new recipe" button. The ingredient input screen appears.

[1199] Step 3:

[1200] The user enters the ingredients they want to use in the text box, entering "chicken," "carrots," and "potatoes" in that order, and taps the "Add" button to add them to the ingredients list.

[1201] Step 4:

[1202] The user enters all ingredients and taps the "Submit" button. The device retrieves the ingredient list and proceeds to the next step.

[1203] Step 5:

[1204] The device converts the ingredient list into JSON format and stores the JSON data of the ingredient list in the HTTP request body.

[1205] Step 6:

[1206] The device sends an HTTP request containing JSON data to the server. The request URL is used to send the data to the server.

[1207] Step 7:

[1208] The server receives the HTTP request and extracts the JSON data of the ingredient list from the request body.

[1209] Step 8:

[1210] The server parses the JSON data of the ingredient list and obtains information about each ingredient: "chicken," "carrot," and "potato."

[1211] Step 9:

[1212] The server passes the list of ingredients as input to the generative AI model, which then generates a recipe based on the ingredients.

[1213] Step 10:

[1214] The server converts the generated recipe information into JSON format, generates an HTTP response, and stores the JSON recipe information in the response body.

[1215] Step 11:

[1216] The server sends an HTTP response including the generated recipe information to the terminal.

[1217] Step 12:

[1218] The device receives the HTTP response and obtains the recipe information JSON data from the response body.

[1219] Step 13:

[1220] The device parses the JSON data and extracts the recipe information for "Chicken and Vegetable Stew." It then generates the recipe screen.

[1221] Step 14:

[1222] The device displays a recipe viewing screen, providing the user with information such as the recipe title, ingredients, and cooking steps.

[1223] Step 15:

[1224] The user checks the displayed recipes and starts cooking based on the recipe "Chicken and Vegetable Stew."

[1225] These are the specific processing steps of this system, which allows users to efficiently use ingredients at home and obtain new cooking recipes.

[1226] Example 1

[1227] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1228] In today's busy lifestyles, users need a way to efficiently utilize ingredients on hand and easily create new recipes. However, conventional systems have complicated processes for inputting ingredients and creating recipes, which is time-consuming for users. In addition, they lack the functionality to create recipes specialized for health management or customized recipes based on user preferences.

[1229] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1230] In this invention, the server includes a means for analyzing the ingredient list and converting it into a prompt statement, a means for inputting the prompt statement into a generative AI model to generate a recipe, and a means for formatting the generated recipe into JSON format and sending it to the terminal as an HTTP response. This allows the user to easily input ingredients on hand and automatically generate a new recipe using the generative AI model. It also simultaneously enables parameter adjustment according to the user's preferences and the generation of recipes specialized for health management.

[1231] 1. A "user" is an individual who uses the system to input ingredients on hand and generate recipes.

[1232] 2. "Terminal" means the collection of hardware and software used by a user to input ingredients, transmit the input to the server, and receive and display the generated recipe.

[1233] 3. "Server" means a collection of hardware and software that analyzes the ingredient list received from the Terminal, generates a recipe using a generative AI model, and sends it to the Terminal.

[1234] 4. An "ingredient list" is a collection of data that a user enters to identify the ingredients they wish to use.

[1235] 5. "JSON format" stands for JavaScript Object Notation and is a standard for representing data in a lightweight text format.

[1236] 6. "HTTP request" is a part of the communication protocol used by a device to send data to a server.

[1237] 7. "HTTP response" is part of the communication protocol used by the server to send data to the terminal.

[1238] 8. A "prompt" is a sentence that is input to the generative AI model to generate a recipe based on a list of ingredients.

[1239] 9. “Generative AI Model” means an artificial intelligence model for generating new recipes based on input prompts.

[1240] 10. "Recipe display screen" is an interface that allows the terminal to visually display to the user recipe information received from the server.

[1241] The above definitions of each word concretely show the functions and roles of the system.

[1242] MODE FOR CARRYING OUT THE INVENTION

[1243] An embodiment of this invention is a system that generates and provides new recipes using ingredients when a user inputs ingredients available at home into an application. Specifically, the system includes a means for the user to input ingredients, a means for the terminal to send the input ingredient list to a server, a means for the server to generate a recipe using a generative AI model, a means for sending the generated recipe to the terminal, and a means for the terminal to display the generated recipe to the user.

[1244] Hardware and software used

[1245] Hardware

[1246] Devices such as smartphones, tablets, and computers

[1247] Server for recipe generation and data processing

[1248] software

[1249] Application for inputting ingredients

[1250] Data communication using the HTTP protocol

[1251] Generative AI models (e.g., OpenAI GPT-3)

[1252] System configuration and operation

[1253] Below, each component of this system and its operation will be explained in detail.

[1254] The user launches the application on their smartphone or tablet and displays the ingredient input screen. The user enters the ingredients they want to use one by one into the text box and taps the "Add" button to add them to the ingredient list. After entering all the ingredients, the user taps the "Submit" button.

[1255] The device obtains the ingredient list entered by the user and converts it to JSON format. The converted JSON data is stored in the body of the HTTP request and an HTTP POST request is sent to the specified endpoint on the server. This request includes the ingredient list entered by the user.

[1256] The server receives the HTTP request and extracts the ingredient list from the request body. It then parses the ingredient list to generate a prompt. For example, it generates a prompt like, "The user has the following ingredients: - chicken - carrots - potatoes. Please generate a new recipe using these ingredients."

[1257] The generated prompt is input into a generative AI model to generate a recipe. The generative AI model generates a new recipe (e.g., "chicken and vegetable stew") and formats the result in JSON format. The formatted JSON data is stored in the body of an HTTP response and sent to the device.

[1258] The device receives the HTTP response from the server and analyzes the JSON data in the response. Based on the analysis results, it creates a recipe display screen and displays the new recipe to the user. The user can then start cooking according to the recipe.

[1259] Specific examples

[1260] For example, if a user has chicken, carrots, and potatoes at home, a new recipe will be generated using those ingredients.

[1261] The user launches the application, enters "chicken," "carrots," and "potatoes," and taps the "Send" button. The device sends the ingredient list to the server, which analyzes the received ingredient list and inputs it into the generative AI model. The generative AI model generates a recipe (e.g., "chicken and vegetable stew"), and the server formats the recipe information in JSON format and sends it to the device. The device receives this recipe information and displays the recipe display screen to the user. The user can then start cooking based on the "chicken and vegetable stew" recipe.

[1262] In this way, users can efficiently use ingredients they have on hand and enjoy new dishes. In addition, by adjusting the parameters of the generative AI model, it is possible to customize recipes according to user preferences and generate recipes specialized for health management.

[1263] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1264] The flow of this system's program processing

[1265] Step 1:

[1266] The user launches the application and the ingredient input screen is displayed. The user enters the ingredients they want to use one by one into the text box and taps the "Add" button. For example, the user can enter "chicken," "carrot," and "potato" in that order. When the user taps the "Submit" button, all the ingredients entered are included in the list.

[1267] Input: User input of ingredients (e.g. "chicken", "carrot", "potato")

[1268] Output: List of ingredients (e.g. ["chicken", "carrot", "potato"])

[1269] Step 2:

[1270] The device converts the ingredient list entered by the user into JSON format and stores it in the body of the HTTP request. This request is then sent to the server. Specifically, the ingredient list is sent to the specified endpoint on the server with the "application / json" content type.

[1271] Input: List of ingredients (e.g. ["chicken", "carrot", "potato"])

[1272] Output: JSON format data and HTTP request (e.g., {"ingredients":["chicken", "carrot", "potato"]})

[1273] Step 3:

[1274] The server analyzes the received HTTP request and retrieves the list of ingredients from the request body. It then uses the retrieved ingredient list to generate a prompt. This prompt is input into the generative AI model and serves as an instruction for generating a recipe. For example, the prompt might be in the form, "The user has the following ingredients: - chicken - carrots - potatoes. Please generate a new recipe using these ingredients."

[1275] Input: HTTP request, JSON format data (e.g., {"ingredients":["chicken", "carrot", "potato"]})

[1276] Output: Prompt text (e.g., "The user has the following ingredients: - chicken - carrots - potatoes. Generate a new recipe using these ingredients.")

[1277] Step 4:

[1278] The server inputs the prompt into a generative AI model to generate a new recipe. A generative AI model (e.g., OpenAI GPT-3) is used to generate a recipe based on the prompt. The generated recipe is then converted back to JSON format and stored in the body of the HTTP response.

[1279] Input: Prompt text (e.g., "The user has the following ingredients: - chicken - carrots - potatoes. Generate a new recipe using these ingredients.")

[1280] Output: Recipe information (e.g., "Chicken and vegetable stew"), JSON-formatted recipe data (e.g., {"Recipe": "Chicken and vegetable stew", "Steps": ["Cut the chicken", "Chop the vegetables", "Simmer"]})

[1281] Step 5:

[1282] The server stores the JSON data of the generated recipe in the HTTP response body and sends it to the terminal.

[1283] Input: JSON-formatted recipe data (e.g., {"Recipe": "Chicken and vegetable stew", "Steps": ["Cut the chicken", "Chop the vegetables", "Simmer"]})

[1284] Output: HTTP response, recipe data in JSON format

[1285] Step 6:

[1286] The device receives the HTTP response from the server and retrieves the JSON-formatted recipe data from the response body. It then analyzes this data to create a recipe display screen and presents it to the user. The user can then check the recipe and start cooking.

[1287] Input: HTTP response, JSON-formatted recipe data (e.g., {"Recipe": "Chicken and vegetable stew", "Steps": ["Cut the chicken", "Chop the vegetables", "Simmer"]})

[1288] Output: Recipe display screen, visual display of recipe information

[1289] The above is the flow of processing in the program of this system and the specific operations performed at each step.

[1290] (Application example 1)

[1291] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1292] While existing systems exist that efficiently utilize ingredients available at home to generate new recipes, they do not include a means for easily obtaining ingredients or seasonings that are in short supply, which means that users have to go shopping separately to obtain ingredients that are not available at home, resulting in inconvenience. This also leads to food waste and limits the variety of dishes that can be made. Therefore, there is a need for the development of a new system that allows users to easily obtain ingredients and seasonings that are in short supply.

[1293] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1294] In this invention, the server includes a means for inputting ingredients, a means for transmitting an ingredient list to the server, a means for generating a recipe using a generative model, a means for transmitting the generated recipe to a terminal, a means for displaying the generated recipe, and a means for the user to order delivery of ingredients and seasonings that the user is running low on. This allows the user to make the most of ingredients they have on hand and efficiently obtain ingredients and seasonings that they are running low on, enabling them to enjoy new variations of dishes.

[1295] "User" refers to an individual who uses this system to input ingredients, create recipes, and place delivery orders.

[1296] "Terminal" refers to an electronic device used by a user, such as a smartphone or computer.

[1297] The "ingredient list" refers to a list of multiple ingredients entered by the user.

[1298] "Server" refers to a computer system that runs on the cloud or a network and generates recipes and communicates data.

[1299] A "generative model" refers to a machine learning algorithm or AI system that creates new recipes based on input data.

[1300] A "recipe" refers to detailed information about cooking steps and ingredients generated using an ingredient list.

[1301] "Delivery order" refers to the act of a user ordering ingredients or seasonings they are running low on via the Internet and having them delivered to their home.

[1302] "Display means" refers to a function for visually showing the recipe generated on the terminal to the user.

[1303] The present invention provides a system that allows a user to make the most of ingredients on hand and efficiently obtain ingredients and seasonings that are in short supply. Specific embodiments of the present invention are described below.

[1304] System Configuration

[1305] The system includes a terminal used by a user (e.g., a smartphone or tablet), a server, and a network that communicates via the Internet.

[1306] Hardware and software used

[1307] Device: Smartphone or tablet (device where user inputs ingredients and displays recipe and delivery order information).

[1308] Server: A computer system in the cloud (implemented using Python and Flask).

[1309] Generative AI models: Generative models such as OpenAI GPT-4.

[1310] Network: A network for transmitting data between devices and servers over the Internet.

[1311] Function and process description

[1312] Enter ingredients

[1313] The user launches the app on their device and inputs the ingredients they have on hand. On the ingredient input screen, the user enters the ingredients they want to use in the text box and taps the "Add" button to add them to the ingredient list.

[1314] Send ingredient list

[1315] The terminal converts the ingredient list entered by the user into JSON format, stores it in the HTTP request body, and sends it to the server.

[1316] Recipe Generation

[1317] The server receives the input ingredient list and analyzes it. The analyzed ingredient list is input into the generative AI model, and a new recipe is generated. The recipe generated by the generative AI model is formatted in JSON and sent to the device as an HTTP response.

[1318] View recipes and order delivery

[1319] The device analyzes the response received from the server and obtains the generated recipe information. The user can then cook based on this information. They can also order delivery of missing ingredients and seasonings based on the generated recipe.

[1320] Specific examples

[1321] For example, if a user has "tomatoes," "olive oil," and "basil" at home, the following processing will occur:

[1322] Example prompt sentence:

[1323] "Suggest a new recipe using the following ingredients: tomatoes, olive oil, and basil."

[1324] Generated recipe example:

[1325] Tomato and Basil Pasta: Finely chop the tomatoes and fry them in olive oil. Add the basil and fry some more, then mix with the cooked pasta. Season with salt and pepper and top with Parmesan cheese.

[1326] The user can start cooking based on this generated recipe information, and can easily order delivery to get any ingredients they are missing.

[1327] This allows users to make the most of the ingredients they have on hand, expanding the variety of dishes they can make, while also reducing food waste.

[1328] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1329] Step 1:

[1330] Enter ingredients

[1331] The user launches the app on their device and displays the ingredient input screen. The user types the ingredients they have at home into the text box and taps the "Add" button. This adds the ingredients to the list. After entering all the ingredients, the user taps the "Submit" button.

[1332] Input: Ingredients entered by the user.

[1333] Output: A list of ingredients is generated.

[1334] Step 2:

[1335] Send ingredient list

[1336] The device converts the ingredient list entered by the user into JSON format, then sends this JSON data to the server as an HTTP POST request.

[1337] Input: A list of ingredients entered by the user.

[1338] Output: A JSON formatted list of ingredients is sent to the server.

[1339] Step 3:

[1340] Recipe Generation

[1341] The server parses the JSON-formatted ingredient list received from the device. The server calls the API of a generative AI model (e.g., OpenAI GPT-4) and provides the ingredient list as input data. The generative AI model generates a new recipe based on the input data. The generated recipe is formatted in JSON format.

[1342] Input: A list of ingredients in JSON format.

[1343] Output: The new recipe generated by the generative AI model.

[1344] Step 4:

[1345] Send recipe

[1346] The server sends the created recipe to the terminal as an HTTP response in JSON format.

[1347] Input: The generated recipe.

[1348] Output: The recipe in JSON format is sent to the terminal.

[1349] Step 5:

[1350] Recipe display

[1351] The device analyzes the JSON response received from the server. The device displays the analyzed recipe information on the recipe screen. The user can check the displayed recipe and start cooking. It is also possible to order delivery of missing ingredients and seasonings based on the recipe.

[1352] Input: Recipe information in JSON format.

[1353] Output: The parsed recipe information is displayed to the user.

[1354] Through these steps, users can not only efficiently utilize the ingredients they have on hand and create new recipes, but also easily order delivery of ingredients or seasonings they are running low on.

[1355] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1356] This invention relates to a system that generates and provides new recipes using ingredients when a user inputs ingredients available at home into an application. Specifically, the system includes a means for the user to input ingredients, a means for the terminal to send the input ingredient list to a server, a means for the server to generate a recipe using a generative model, a means for the terminal to send the generated recipe, a means for the terminal to display the generated recipe to the user, and an emotion engine that recognizes the user's emotions and adjusts the parameters of a recipe generation model based on the emotions.

[1357] System program processing overview

[1358] The system is programmed as follows:

[1359] Ingredients input stage

[1360] The user launches the app and the ingredient input screen is displayed. The user enters the ingredients they want to use and taps the Add button to add them to the ingredient list. After entering all ingredients, they tap the Send button.

[1361] Ingredient list sending stage

[1362] The device sends the ingredient list entered by the user to the server. The device converts the ingredient list into JSON format, stores it in the HTTP request body, and sends it to the server.

[1363] Emotion recognition stage by server

[1364] The server receives the HTTP request and extracts the JSON data of the ingredient list from the request body. Next, the emotion engine recognizes the user's emotion. Emotions are acquired and analyzed using sensor data from cameras, microphones, etc.

[1365] Recipe Generation Stage

[1366] The server analyzes the ingredient list and adjusts the parameters of the generative AI model based on the user's perceived emotions. For example, if the user is feeling stressed, the model is adjusted to prioritize ingredients and cooking methods that have a relaxing effect. The server then inputs the ingredient list into the generative AI model to generate a recipe.

[1367] Recipe submission stage

[1368] The server converts the generated recipe information into JSON format and sends it to the terminal as an HTTP response.

[1369] Recipe display stage

[1370] The terminal analyzes the response received from the server and acquires the recipe information. The terminal displays a recipe viewing screen and provides the created recipe to the user.

[1371] Specific examples

[1372] For example, if a user wants to get new recipes using "chicken," "carrots," and "potatoes," he or she will use this system.

[1373] The user inputs ingredients into the app, entering "chicken," "carrots," and "potatoes" respectively. When the user taps the send button, the device sends the list of ingredients to the server.

[1374] The server analyzes the received ingredient list and also recognizes the user's emotions from camera and microphone data. For example, if the server recognizes that the user is tired, the generative AI model generates a recipe with an emphasis on "dishes that have a refreshing effect." The generated recipe is "chicken and carrot soup."

[1375] The server converts this recipe information into JSON format and sends it to the device.

[1376] The device receives the recipe information and shows the recipe display screen to the user, who then begins cooking based on the recipe for "chicken and carrot soup."

[1377] In this way, the system can generate optimal recipes based on the user's emotions, improving the user's cooking experience by providing new dishes without wasting ingredients on hand.

[1378] The processing flow will be explained below.

[1379] Step 1:

[1380] The user launches the app and the home screen appears.

[1381] Step 2:

[1382] The user taps the "Create a new recipe" button. The ingredient input screen appears.

[1383] Step 3:

[1384] The user enters the ingredients they want to use in the text box, entering "chicken," "carrots," and "potatoes" in that order, and taps the "Add" button to add them to the ingredients list.

[1385] Step 4:

[1386] The user enters all ingredients and taps the "Submit" button. The device retrieves the ingredient list and proceeds to the next step.

[1387] Step 5:

[1388] The device converts the ingredient list into JSON format and stores the JSON data of the ingredient list in the HTTP request body.

[1389] Step 6:

[1390] The device sends an HTTP request containing JSON data to the server. The request URL is used to send the data to the server.

[1391] Step 7:

[1392] The server receives the HTTP request and extracts the JSON data of the ingredient list from the request body.

[1393] Step 8:

[1394] The server parses the JSON data of the ingredient list and obtains information about each ingredient: "chicken," "carrot," and "potato."

[1395] Step 9:

[1396] The server activates the emotion engine to recognize the user's emotions. It acquires sensor data from cameras and microphones and analyzes the user's facial expressions and tone of voice.

[1397] Step 10:

[1398] The server determines the user's emotion from the analysis results of the emotion engine. For example, it determines that the user is tired.

[1399] Step 11:

[1400] The server adjusts the parameters of the generative AI model based on the analysis results of the emotion engine, for example, by changing the settings to prioritize dishes with a refreshing effect.

[1401] Step 12:

[1402] The server passes the list of ingredients as input to the generative AI model and generates a recipe, for example, "chicken and carrot soup."

[1403] Step 13:

[1404] The server converts the generated recipe information into JSON format, generates an HTTP response, and stores the JSON recipe information in the response body.

[1405] Step 14:

[1406] The server sends an HTTP response including the generated recipe information to the terminal.

[1407] Step 15:

[1408] The device receives the HTTP response and obtains the recipe information JSON data from the response body.

[1409] Step 16:

[1410] The device parses the JSON data and extracts the recipe information for "Chicken and Carrot Soup." It then generates the recipe screen.

[1411] Step 17:

[1412] The device displays a recipe viewing screen, providing the user with information such as the recipe title, ingredients, and cooking steps.

[1413] Step 18:

[1414] The user checks the displayed recipes and starts cooking based on the "Chicken and Carrot Soup" recipe.

[1415] These are the specific processing steps in this system, which allows users to obtain the best recipes based on the ingredients and emotions at home.

[1416] Example 2

[1417] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1418] Conventional recipe generation systems are unable to generate recipes that reflect the user's current emotions and preferences, and the recipes they provide may not meet the user's expectations or needs. Furthermore, they have issues with being unable to efficiently utilize the ingredients the user has on hand, and not proposing recipes that are appropriate from a health management perspective.

[1419] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for recognizing a user's emotion using an emotion recognition engine, a means for adjusting parameters of a generative AI model based on the recognized emotion and generating a recipe, and a means for transmitting the generated recipe to a terminal. This makes it possible to provide an optimal recipe according to the user's emotion and preferences.

[1420] "User" refers to a person who uses the system to input ingredients and obtain recipes.

[1421] A "terminal" is a device that allows a user to input ingredients, transmits an ingredient list to a server, and displays the recipe received from the server.

[1422] The "server" is a computer system that receives the ingredient list sent by the user, recognizes the user's emotions using an emotion recognition engine, generates a recipe using a generative AI model, and sends the generated recipe to the terminal.

[1423] An "emotion recognition engine" is a system or algorithm that analyzes data obtained from the user's camera or microphone to recognize the user's emotions.

[1424] A "generative AI model" is a machine learning model that automatically generates new recipes based on input data, and is particularly capable of adjusting parameters according to the user's emotions.

[1425] The "ingredient list" is a list of multiple ingredients that the user wants to use, and is data in a format that is sent from the terminal to the server.

[1426] A "recipe" is information that provides detailed instructions for making a dish using ingredients, and is generated by a generative AI model.

[1427] "Parameter adjustment" refers to changing the internal settings of a generative AI model to change its behavior depending on specific conditions or situations.

[1428] This invention relates to a system that generates and provides new recipes using ingredients that a user has at home when they input them into an application. This system is composed of the following specific hardware and software.

[1429] Hardware and software used

[1430] The terminal used by the user is a mobile device such as a smartphone or tablet, and an application for inputting ingredients is installed on this terminal.

[1431] The server is a typical server providing cloud computing services, and includes web server software (e.g., Apache or Nginx) to process HTTP requests, software to run a generative AI model (e.g., GPT-4) that analyzes ingredient lists and generates recipes, and image and voice analysis technologies required to run an emotion recognition engine.

[1432] Examples of user actions

[1433] When a user launches the app, an ingredient input screen appears. This screen has a text box and an "Add" button and a "Submit" button. The user inputs ingredients such as "chicken," "carrots," and "potatoes," and taps the "Add" button to add them to the ingredient list. Once all ingredients have been input, the user taps the "Submit" button to confirm the list.

[1434] Specific examples of terminal operation

[1435] The device sends the ingredient list entered by the user to the server. At this time, the ingredient list is converted to JSON format and sent to the server as the body of the HTTP request.

[1436] Example of server operation

[1437] When the server receives an HTTP request, it extracts the JSON data from the request body, parses the list of ingredients, and uses an emotion recognition engine to recognize the user's emotion. For example, it can recognize the user's emotion of "tired" through sensor data collected from the camera and microphone.

[1438] Next, the server adjusts the parameters of the generative AI model based on the recognized user's emotions and generates a recipe. For example, a prompt commanding the model to generate a "refreshing recipe for a tired user" is input. Suppose the generated recipe is "chicken and carrot soup."

[1439] "Please suggest a refreshing dish using chicken, carrots, and potatoes for tired users."

[1440] Use a prompt such as:

[1441] The server converts the generated recipe information into JSON format and sends it to the terminal as an HTTP response. The terminal analyzes the received response and obtains the generated recipe information (e.g., "Chicken and Carrot Soup"). Finally, the terminal displays the recipe viewing screen to the user.

[1442] In this way, the system can provide optimal recipes based on the user's emotions and preferences, providing a new cooking experience without wasting the ingredients the user has.

[1443] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1444] Step 1:

[1445] The user launches the app and displays the ingredient input screen.

[1446] Specific operation: The user taps the app icon on their smartphone or tablet to launch the application. After launching, the ingredient input screen is displayed.

[1447] Input and output: The input is a user operation (tapping the app icon), and the output is the display of the ingredient input screen.

[1448] Step 2:

[1449] The user enters the ingredients they want to use and taps the add button.

[1450] Specific behavior: The user enters "chicken" in the text box and taps the "Add" button. This displays the ingredients in the text box as a list. This operation is repeated for "carrots" and "potatoes."

[1451] Input and output: The input is the ingredient name and tapping the add button, and the output is the display of the ingredient list.

[1452] Step 3:

[1453] After the user enters all the ingredients, he taps the submit button.

[1454] Specific actions: Once the user has entered all the required ingredients, they tap the "Submit" button on the screen.

[1455] Input and Output: The input is the tap of the send button, and the output is the trigger of the send request.

[1456] Step 4:

[1457] The terminal transmits the ingredient list input by the user to the server.

[1458] Specific operation: The device serializes the input ingredient list into JSON format, stores it in the HTTP request body, and sends it to the server.

[1459] Input and Output: The input is the list of ingredients entered by the user, and the output is the HTTP request sent to the server.

[1460] Step 5:

[1461] The server receives the HTTP request and extracts the JSON data of the ingredient list from the request body.

[1462] Specific behavior: The server parses the received HTTP request and extracts a JSON-formatted list of ingredients from the request body.

[1463] Input and Output: The input is the HTTP request sent from the terminal, and the output is the parsed JSON data of the ingredient list.

[1464] Step 6:

[1465] The server uses an emotion recognition engine to recognize the user's emotion.

[1466] Specific operation: The server analyzes sensor data obtained from the camera and microphone and recognizes the user's emotions. For example, it detects emotions such as "tired" from the user's facial expression and tone of voice.

[1467] Input and Output: The input is the sensor data and the ingredient list, and the output is the recognized user emotion data.

[1468] Step 7:

[1469] The server adjusts the parameters of the generative AI model based on the recognized emotions and generates a recipe.

[1470] Specific operation: The server generates a prompt sentence based on the emotion recognition result and inputs it into a generative AI model (e.g., GPT-4). Let's assume that the generated recipe is "chicken and carrot soup."

[1471] Input and Output: The input is the recognized emotion data and the ingredients list, and the output is the generated recipe.

[1472] Step 8:

[1473] The server converts the generated recipe information into JSON format and sends it to the terminal as an HTTP response.

[1474] Specific operation: The server serializes the generated recipe information in JSON format, stores it in the body of the HTTP response, and sends it to the terminal.

[1475] Input and Output: The input is the generated recipe information, and the output is the HTTP response sent to the terminal.

[1476] Step 9:

[1477] The terminal analyzes the response received from the server and acquires the recipe information.

[1478] Specific operation: The device parses the HTTP response received and extracts the recipe information.

[1479] Input and Output: The input is the HTTP response received from the server, and the output is the extracted recipe information.

[1480] Step 10:

[1481] The terminal displays a recipe viewing screen and provides the generated recipe to the user.

[1482] Specific operation: The device displays a recipe viewing screen and provides the user with detailed information such as ingredients, steps, and tips for the dish.

[1483] Input and Output: The input is the extracted recipe information and the output is the recipe screen that is displayed to the user.

[1484] This system allows users to efficiently use ingredients they have on hand and obtain the optimal recipe based on their emotional state.

[1485] (Application example 2)

[1486] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1487] Conventional recipe generation systems only generate recipes based on the ingredients a user has on hand, and do not consider the user's emotions or mood when suggesting recipes. Furthermore, they lack a function for instantly ordering food based on the generated recipe, forcing users to search for purchasing locations and ordering methods themselves based on the suggested recipe. This results in a poor user experience and a lack of convenience. To solve these problems, a system is needed that generates recipes that consider the user's emotions and seamlessly processes food orders from external organizations.

[1488] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1489] In this invention, the server includes a means for recognizing a user's emotion, a means for adjusting parameters of a generative model based on the recognized emotion, and a means for the terminal to order food from an external organization based on a recipe, thereby enabling the generation and suggestion of optimal recipes tailored to the user's emotion, and further enabling the user to quickly and conveniently order food based on the recipe.

[1490] "User" refers to an individual who uses the system to input ingredients, make recipe suggestions, and place food orders.

[1491] "Ingredients" are food items owned by the user that are the basis for recipes.

[1492] "Terminal" refers to a device operated by a user, such as a smartphone, tablet, or PC.

[1493] A "server" is a remote computer that receives and processes data sent from a terminal.

[1494] A "generative model" refers to an algorithm that analyzes input data and generates a recipe.

[1495] A "recipe" refers to the cooking steps and methods generated by the system based on ingredients.

[1496] "Emotion recognition" refers to technology that analyzes and recognizes a user's emotions based on sensor data from cameras, microphones, etc.

[1497] "Parameter adjustment" refers to changing the settings of a generative model based on user emotions and other data.

[1498] "External organization" refers to a restaurant or grocery store that is external to the system and provides the food ordered by the user.

[1499] The system of the present invention is a system that generates new recipes based on ingredients that the user has, and then allows the user to order food based on the recipes. Specific embodiments for carrying out the invention are described below.

[1500] The system consists of the following components:

[1501] Terminal: A device operated by a user, such as a smartphone, tablet, or PC. The terminal is used to input ingredients, display recipes, and order food through an application.

[1502] Server: A remote computer that receives user-entered data and performs emotion recognition and recipe generation. Here, we can use Python and Flask as the backend server.

[1503] Generative model: An algorithm that analyzes input data and generates recipes. Recipes are suggested using generative AI models.

[1504] Emotion recognition engine: A technology that analyzes user emotions based on sensor data from cameras, microphones, etc. It uses libraries such as EmotionRecognizer.

[1505] External Organization: An organization outside the system that provides the food that users order, such as a restaurant or grocery store.

[1506] Example of a system

[1507] Step 1: Enter ingredients

[1508] The user launches the application and displays the ingredient input screen. The user enters the ingredients they have on hand and taps the Add button to add them to the list. After entering all the ingredients, they tap the Send button, and the ingredient list is sent from the device to the server in JSON format.

[1509] Step 2: Emotion Recognition

[1510] The server receives the ingredient list entered by the user and simultaneously acquires sensor data from the camera and microphone. The emotion recognition engine analyzes this sensor data and recognizes the user's emotions.

[1511] Step 3: Recipe generation

[1512] The server adjusts the parameters of the generative AI model based on the emotional data obtained by the emotion recognition engine. For example, if the user feels like relaxing, the server adjusts the generative model to prioritize ingredients and cooking methods that have a relaxing effect. The server then inputs the ingredient list into the generative AI model and generates a new recipe.

[1513] Step 4: View recipes and order

[1514] The generated recipe information is converted back to JSON format and sent from the server to the device. The device parses the received recipe information, displays it to the user, and provides the user with the option to order food from an external organization (e.g., a nearby restaurant) based on this recipe.

[1515] Examples of prompt statements

[1516] By inputting prompts to the generative AI model as follows, a recipe based on the user's emotions is generated:

[1517] The user wants to relax. Generate a relaxing recipe using the following ingredients:

[1518] chicken meat

[1519] Carrots

[1520] potatoes

[1521] The system embodying this invention allows users to receive optimal recipe suggestions based on their emotions and quickly order food based on those recipes, thereby improving the user experience and dramatically increasing convenience.

[1522] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1523] Step 1:

[1524] The user launches the application. The user accesses the ingredient input screen and inputs the ingredients they have on hand. After inputting each ingredient, they tap the Add button to add it to the list. After inputting all ingredients, they tap the Send button. This saves the input ingredient list to the device. The input data is the name of the specific ingredients the user has in their hand.

[1525] Step 2:

[1526] The device sends the ingredient list to the server. The device converts the saved ingredient list into JSON format, stores it in the HTTP request body, and sends it to the server. The sent data is the ingredient list in JSON format. The server receives this request.

[1527] Step 3:

[1528] The server parses the received ingredient list. The server extracts the JSON data from the HTTP request body and defines the ingredient list as a data structure. This data is the parsed ingredient list.

[1529] Step 4:

[1530] The server recognizes the user's emotions. The server activates an emotion recognition engine using sensor data acquired from the device, such as a camera or microphone. The sensor data includes facial expressions and tone of voice, and is analyzed using a library such as EmotionRecognizer. The result of this analysis is the recognized user's emotion data.

[1531] Step 5:

[1532] The server adjusts the parameters of the generative model based on the emotion recognition results. The server adaptively changes the parameters of the generative AI model based on the recognized emotion data. For example, when a user is feeling stressed, the server sets the parameters to prioritize recipes that have a relaxing effect. These adjusted parameters are used as input for the generative model.

[1533] Step 6:

[1534] The server generates a recipe using the generative AI model. The server inputs the adjusted parameters and ingredient list into the generative AI model to generate a new recipe. The output data is the generated recipe information.

[1535] Step 7:

[1536] The server converts the generated recipe information into JSON format and sends it to the terminal as an HTTP response. The response data contains detailed information about the generated recipe. The terminal receives this response.

[1537] Step 8:

[1538] The device analyzes the received recipe information and displays it to the user. The generated recipe is displayed on the recipe browsing screen. The device also provides an option to order food from an external organization based on the recipe. For example, it displays a link or button to order directly from a restaurant or grocery store. This allows the user to quickly order food based on the suggested recipe.

[1539] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1540] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1541] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1542] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1543] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1544] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1545] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1546] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1547] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1548] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1549] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1550] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1551] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1552] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1553] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1554] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1555] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1556] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1557] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1558] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1559] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1560] The following is further disclosed regarding the above embodiment.

[1561] (Claim 1)

[1562] A means for a user to input ingredients;

[1563] a means for the terminal to transmit the ingredient list input by the user to the server;

[1564] a means for the server to generate recipes using the generative model;

[1565] A means for transmitting the recipe generated by the server to the terminal;

[1566] means for the terminal to display the generated recipe to the user;

[1567] A system including:

[1568] (Claim 2)

[1569] 2. The system according to claim 1, wherein the recipe generating means adjusts parameters of the generative model according to user preferences.

[1570] (Claim 3)

[1571] 2. The system according to claim 1, wherein the recipe generating means generates a recipe specialized for health management.

[1572] "Example 1"

[1573] (Claim 1)

[1574] A means for a user to input ingredients;

[1575] a means for the terminal to transmit the ingredient list input by the user to the server;

[1576] The device converts the ingredient list into JSON format and sends it to the server as an HTTP request.

[1577] A means for the server to parse the received ingredient list and convert it into a prompt sentence;

[1578] A means for the server to input a prompt sentence into a generative AI model and generate a recipe;

[1579] The server formats the recipes it generates into JSON format and sends them to the terminal as an HTTP response.

[1580] a means for the terminal to analyze a response from the server, acquire recipe information, and display it to the user;

[1581] A system including:

[1582] (Claim 2)

[1583] 2. The system according to claim 1, wherein the recipe generation means adjusts parameters of the generation AI model according to user preferences.

[1584] (Claim 3)

[1585] 2. The system according to claim 1, wherein the recipe generating means generates a recipe specialized for health management.

[1586] "Application Example 1"

[1587] (Claim 1)

[1588] A means for a user to input ingredients;

[1589] a means for the terminal to transmit the ingredient list input by the user to the server;

[1590] a means for the server to generate recipes using the generative model;

[1591] A means for transmitting the recipe generated by the server to the terminal;

[1592] means for the terminal to display the generated recipe to the user;

[1593] A way for users to order delivery of ingredients and seasonings they are running low on,

[1594] A system including:

[1595] (Claim 2)

[1596] 2. The system according to claim 1, wherein the recipe generating means adjusts parameters of the generative model according to user preferences.

[1597] (Claim 3)

[1598] 2. The system according to claim 1, wherein the recipe generating means generates a recipe specialized for health management.

[1599] "Example 2: Combining Emotion Engines"

[1600] (Claim 1)

[1601] A means for a user to input ingredients;

[1602] a means for the terminal to transmit the ingredient list input by the user to the server;

[1603] A means for the server to recognize the user's emotion using an emotion recognition engine;

[1604] A means for the server to adjust parameters of the generative AI model based on the recognized emotion and generate a recipe;

[1605] A means for transmitting the recipe generated by the server to the terminal;

[1606] means for the terminal to display the generated recipe to the user;

[1607] A system including:

[1608] (Claim 2)

[1609] 2. The system according to claim 1, wherein the recipe generation means adjusts parameters of the generation AI model according to user preferences.

[1610] (Claim 3)

[1611] 2. The system according to claim 1, wherein the recipe generating means generates a recipe specialized for health management based on the user's emotions.

[1612] "Application example 2 when combining emotion engines"

[1613] (Claim 1)

[1614] A means for a user to input ingredients;

[1615] a means for the terminal to transmit the ingredient list input by the user to the server;

[1616] a means for the server to generate recipes using the generative model;

[1617] A means for the server to recognize the user's emotion;

[1618] a means for adjusting parameters of a generative model based on the emotion recognized by the server;

[1619] A means for transmitting the recipe generated by the server to the terminal;

[1620] means for the terminal to display the generated recipe to the user;

[1621] means for the terminal to order food from an external organization based on the recipe;

[1622] A system including:

[1623] (Claim 2)

[1624] 2. The system according to claim 1, wherein the recipe generating means adjusts parameters of the generative model according to user preferences.

[1625] (Claim 3)

[1626] 2. The system according to claim 1, wherein the recipe generating means generates a recipe specialized for health management. [Explanation of symbols]

[1627] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for a user to input ingredients; a means for the terminal to transmit the ingredient list input by the user to the server; a means for the server to generate recipes using the generative model; A means for transmitting the recipe generated by the server to the terminal; means for the terminal to display the generated recipe to the user; A system including:

2. 2. The system according to claim 1, wherein the recipe generating means adjusts parameters of the generative model in accordance with user preferences.

3. 2. The system according to claim 1, wherein the recipe generating means generates a recipe specialized for health management.

Images