System

Abstract

A system is provided.SOLUTION: The system includes a user terminal that photographs surplus foodstuff, a means that transmits the photographed image to a server, a generation AI means on the server that registers foodstuff information in a database by analyzing the transmitted image, a server means that transmits a collection instruction to a delivery person of food delivery based on the foodstuff information, a delivery person means that collects the foodstuff according to the delivery instruction and transports the foodstuff to a cooking facility of a partner, a generation AI means on the server that creates an optimal cooking menu based on the foodstuff information, a means that cooks and preserves the foodstuff in the cooking facility of the partner, and a delivery person means that delivers a cooked food based on an order of a user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The problem that this invention aims to solve is to effectively eliminate food waste by efficiently collecting surplus ingredients and using them to prepare meals. Conventional methods make it difficult for individuals and businesses to process surplus ingredients individually, and also make it difficult to efficiently collect and manage them. Therefore, there is a need for a system that comprehensively manages the collection, classification, cooking, and delivery of surplus ingredients. [Means for solving the problem]

[0005] The present invention solves the above problems by providing a system including a user terminal for photographing surplus ingredients, a means for sending the photographed image to a server, a generation AI means on the server for analyzing the sent image and registering ingredient information in a database, a server means for sending collection instructions to a food delivery person based on the ingredient information, a delivery person means for collecting the ingredients in accordance with the delivery instructions and transporting them to an affiliated cooking facility, a generation AI means for creating an optimal dish menu based on the ingredient information in the database, a means for cooking and storing the ingredients at the affiliated cooking facility, and a delivery person means for delivering cooked food based on a user's order.

[0006] "Surplus food" refers to food that is not planned for use by a household or business and may be discarded.

[0007] A "user terminal" refers to a mobile device such as a smartphone or tablet used by a user.

[0008] A "server" is a computer that processes and manages data in a cloud environment or on a network.

[0009] "Generative AI" refers to software that uses artificial intelligence (AI) to automatically perform specific tasks.

[0010] A "database" is a system for storing data in an organized manner and enabling efficient retrieval and updating.

[0011] "Food delivery" refers to a service that delivers food to consumers.

[0012] "Delivery Person" means a person who delivers food to a designated location in order to provide food delivery services.

[0013] A "cooking facility" is a facility for processing and preserving ingredients into dishes.

[0014] A "cooking menu" is a list of dishes and their recipes that can be made using specific ingredients.

[0015] An "order" is when a user indicates their intention to purchase a particular service or product. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The system of the present invention provides a comprehensive solution to eliminate food waste through the efficient collection, management, cooking, and delivery of surplus food ingredients. The detailed process and specific embodiments are described below.

[0038] 1. Take a photo of your surplus food and send it to us

[0039] Users use a smartphone or other device to take a photo of surplus food items using a dedicated application. For example, they can take a photo of leftover tomatoes, peppers, onions, or other ingredients at home. After taking the photo, they can use the application's sending function to send the image to the server.

[0040] 2. Image analysis and database registration

[0041] The server uses a generation AI to analyze the received images. This generation AI uses image recognition technology to determine the type, size, quantity, and freshness of the ingredients. For example, it identifies a red, round object as a "tomato" and a green, elongated object as a "bell pepper." The analyzed ingredient information is then registered in a database.

[0042] 3. Instructions for collection and collection of ingredients

[0043] The server determines the optimal collection schedule and location based on the food ingredient information registered in the database. Collection instructions are sent to the food delivery person, informing them of the specific collection date, time, and location (e.g., User A's address). The delivery person visits the user's address at the specified date and time to collect the excess ingredients.

[0044] 4. Transporting ingredients and delivering them to cooking facilities

[0045] The delivery staff will then transport the collected ingredients to a partner cooking facility, where they will be inspected and cleaned.

[0046] 5. Create a food menu

[0047] The AI ​​on the server creates the optimal menu based on the ingredient information registered in the database. For example, it generates a recipe for "stir-fried vegetables with tomatoes, peppers, and onions." The generated menu and recipe are then sent to the cooking facility.

[0048] 6. Cooking and storage

[0049] The kitchen staff prepares the ingredients according to the recipes created by the generative AI, and once cooking is complete, the food is stored in the appropriate way, such as refrigerated or frozen.

[0050] 7. User Orders and Delivery

[0051] Users can order pre-cooked meals through a dedicated application. The application displays a list of available menu items, allowing users to select the desired dish and complete the order. The order information is sent to a server, which then sends delivery instructions to a delivery person. The delivery person collects the ordered food from the cooking facility and delivers it to the location specified by the user.

[0052] As a specific example, a user takes a photo of two tomatoes, one green pepper, and one onion that they have left over at home and sends it to the server via the app. The server's generation AI analyzes the photo and registers "two tomatoes, one green pepper, and one onion" in the database. A delivery person receives collection instructions, goes to the user's home to collect the ingredients, and transports them to a cooking facility. The generation AI suggests "stir-fried vegetables," and the dish is prepared and stored at the cooking facility. Finally, the user orders the menu item via the app, and the delivery person delivers it to their home. This makes effective use of surplus ingredients and effectively eliminates food waste.

[0053] The processing flow will be explained below.

[0054] Step 1:

[0055] The user launches the app and takes a photo of the surplus food items, such as tomatoes, peppers, or onions, using the camera function on their smartphone.

[0056] Step 2:

[0057] The device sends the captured image to the server. When the image data is sent, it is given a timestamp and user ID information.

[0058] Step 3:

[0059] The server receives the image and analyzes it using generative AI. Specifically, it uses an image recognition algorithm to determine the type, size, quantity, and freshness of the food. For example, it identifies a red, round object as a "tomato" and evaluates its freshness based on its size and color.

[0060] Step 4:

[0061] The server registers the analysis results in a database, recording detailed information such as the freshness of the ingredients (e.g., "2 tomatoes, 1 bell pepper, 1 onion") and the date and time the photo was taken.

[0062] Step 5:

[0063] The server determines the optimal collection schedule and location. Based on the information in the database, it sends collection instructions to the delivery person. These instructions include the date, time, and location of the food collection (e.g., User A's address).

[0064] Step 6:

[0065] The delivery person will collect the ingredients at the specified date, time and location. The delivery person will go to User A's address and pick up the surplus ingredients.

[0066] Step 7:

[0067] The delivery person transports the collected ingredients to the cooking facility. During the transport, the condition of the ingredients is easily checked and any problems are reported to the server.

[0068] Step 8:

[0069] The server communicates ingredient information to the cooking facility, including the type, quantity, and freshness of the ingredients.

[0070] Step 9:

[0071] The generative AI creates the optimal menu, generating recipes such as "stir-fried vegetables with tomatoes, peppers, and onions" based on the ingredients in the database, and sending them to the cooking facility.

[0072] Step 10:

[0073] The staff at the kitchen will prepare the food according to the recipes generated by the AI, and once the food is cooked, it will be stored appropriately, for example by refrigerating or freezing it.

[0074] Step 11:

[0075] Users order pre-cooked meals through a dedicated app, which displays available menu items, and users can select the desired dish to complete the order.

[0076] Step 12:

[0077] The server sends delivery instructions to the delivery person based on the order information. The delivery person is instructed to "Deliver the food from the cooking facility to User B's address."

[0078] Step 13:

[0079] The delivery person picks up the ordered food at the cooking facility and delivers it to the address of User B. After confirming that the food has been delivered safely, the delivery is reported to the server as completed.

[0080] Example 1

[0081] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0082] Currently, the effective use of surplus ingredients and the problem of food waste are social issues, but efficient systems for collection, management, cooking, and delivery are not fully established. There is also a lack of advanced analytical technology to determine the freshness of ingredients and propose appropriate cooking menus. For this reason, a system that can effectively utilize surplus ingredients is needed.

[0083] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0084] In this invention, the server includes a user terminal means for taking pictures of surplus ingredients, a means for sending the taken images to the server, and a generating AI means on the server for analyzing the sent images and registering ingredient information in a database, thereby enabling efficient collection, analysis, and management of surplus ingredients.

[0085] The server also includes a means for sending a collection instruction to a delivery person based on the ingredient information, and a delivery person means for collecting the ingredients according to the collection instruction and transporting them to the cooking facility, thereby enabling the ingredients to be collected and delivered to the cooking facility quickly.

[0086] Furthermore, the server includes a generation AI means for creating an optimal menu based on the ingredient information in the database, a means for cooking and storing ingredients at a cooking facility, a delivery staff means for delivering cooked food based on the user's order, a means for reporting the status of the food en route to the cooking facility to the server in real time, and a means for updating the server with information on completion of storage, thereby enabling the effective use of ingredients, appropriate cooking and storage, and prompt delivery to the user.

[0087] "Surplus ingredients" refers to ingredients or food that are left over and not consumed.

[0088] "User terminal means" refers to a device used by a user, such as a computer or smartphone, that has the function of photographing ingredients and transmitting information about them.

[0089] "Server" refers to a computer system that processes, stores, and transmits data over a network.

[0090] "Generative AI means" refers to technologies that use artificial intelligence (AI) to analyze data and generate specific functions or results.

[0091] "Delivery means" refers to the person or system that collects ingredients or food from a designated location and transports them to a designated location.

[0092] "Ingredient information" refers to data including details such as the type, quantity, and freshness of ingredients.

[0093] A "database" is a data collection point and a system for organizing, storing, and searching various information.

[0094] "Cooking facility" refers to a facility that receives, cooks, and preserves food ingredients.

[0095] "Food Menu" means a list of the types of food available and their details.

[0096] "Freshness" refers to the degree of freshness of food or ingredients, and is usually a standard for evaluating quality, expiration dates, etc.

[0097] "Preservation measures" refers to measures such as refrigeration, freezing, and packaging that are used to store cooked food and ingredients in an appropriate state.

[0098] The system of the present invention provides a comprehensive solution to eliminate food waste through the efficient collection, management, cooking, and delivery of surplus ingredients. Specific embodiments of the system of the present invention are described below.

[0099] Using a smartphone or other user device, a user can take a photo of surplus food items using a dedicated application. For example, they can take a photo of leftover tomatoes, peppers, onions, or other ingredients at home, and press the send button on the application to send the image to the server.

[0100] The server launches a generative AI model to analyze the received images. This generative AI model uses image recognition technologies such as TENSORFLOW (registered trademark) and PyTorch to determine the type, size, quantity, and freshness of ingredients and registers them in a database. For example, it identifies a red, round object as a "tomato" and a green, elongated object as a "bell pepper."

[0101] Based on the food ingredient information registered in the database, the server determines the optimal collection schedule and location. Collection instructions are sent to the delivery person's device, and the delivery person visits the user's address at the specified date and time to collect the excess food. The GPS-equipped device provides the delivery person with the optimal route, supporting efficient collection.

[0102] The delivery person transports the collected ingredients to the cooking facility. During this process, the status of the delivery is reported to the server in real time. Upon arrival at the cooking facility, information confirming the arrival of the ingredients is sent to the server. The cooking facility staff checks the ingredients and cleans up if necessary.

[0103] The generative AI model on the server generates the optimal dish menu based on the ingredient information registered in the database. This process uses a natural language generation model (e.g., GPT-4 (registered trademark)). For example, a recipe for "stir-fried vegetables with tomatoes, peppers, and onions" is generated and sent to the cooking facility.

[0104] The staff at the kitchen prepares the ingredients according to the recipe created by the generative AI. Once the food is cooked, it is stored in an appropriate manner, such as refrigerated or frozen. This information about the completion of storage is updated to the server.

[0105] Users can order pre-cooked meals through a dedicated application. The application displays a list of available menu items, and users can select the desired dish to complete the order. The order information is sent to a server, which then sends delivery instructions to a delivery person. The delivery person collects the ordered food from the cooking facility and delivers it to the address specified by the user.

[0106] Examples:

[0107] The user takes a photo of two surplus tomatoes, one bell pepper, and one onion at home and sends it to the server via the app. The server's generation AI analyzes the photo and registers it in the database as "two tomatoes, one bell pepper, and one onion." A delivery person receives collection instructions, goes to the user's home to collect the ingredients, and transports them to a cooking facility. The generation AI suggests a recipe for "stir-fried vegetables," and the dish is prepared at the cooking facility based on this recipe and stored. Finally, the user orders the menu item via the app, and the delivery person delivers it to their home. This makes effective use of surplus ingredients and effectively eliminates food waste.

[0108] Example prompt for a generative AI model:

[0109] "Please suggest the best recipe based on the ingredient information received from the user. For example, if two tomatoes, one bell pepper, and one onion are provided, please consider what kind of dish can be made using them and provide a detailed recipe."

[0110] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0111] Step 1:

[0112] The user uses a dedicated smartphone application to take a photo of surplus ingredients. For example, they can take a photo of leftover tomatoes, peppers, onions, and other ingredients at home. The input is an image of each ingredient, and the output is an image file.

[0113] Step 2:

[0114] The user presses the send button in the dedicated application to send the captured image to the server. The input is the image file obtained in step 1, and the output is a notification that the image file has been uploaded to the server.

[0115] Step 3:

[0116] The server launches a generative AI model to analyze the received image. The input is the transmitted image file, and the generative AI model performs image recognition using TensorFlow or PyTorch. The output is the recognized ingredient information (e.g., 2 tomatoes, 1 bell pepper, 1 onion). The server registers this information in a database.

[0117] Step 4:

[0118] The server determines the optimal collection schedule and location based on the food ingredient information registered in the database. The input is the food ingredient information registered in the database, and the output is collection instructions to the delivery person. The collection instructions include the specific collection date, time, and location.

[0119] Step 5:

[0120] The delivery person visits the user's address at the specified date and time to collect the excess ingredients. The input is the collection instruction from the server, and the output is the collected ingredients. The delivery person's device is equipped with a GPS, which provides the optimal route.

[0121] Step 6:

[0122] The delivery person transports the collected ingredients to the cooking facility and reports the status of the delivery to the server in real time. The input is the collected ingredients, and the output is a notification of arrival at the cooking facility.

[0123] Step 7:

[0124] The kitchen staff checks the food as it arrives and cleans it up as necessary. The input is the delivered food, and the output is the cleaned food.

[0125] Step 8:

[0126] A generative AI model on the server generates optimal cooking menus based on ingredient information registered in a database. The input is ingredient information from the database, and the generative AI model uses natural language generation technologies such as GPT-4. The output is the generated cooking menu and recipe.

[0127] Step 9:

[0128] The kitchen staff prepares the ingredients according to the recipes created by the generative AI model. The input is the generated food menu and recipe, and the output is the cooked food.

[0129] Step 10:

[0130] Once the food is cooked, it is stored in an appropriate way, such as refrigerated or frozen. The input is the cooked food, and the output is the stored food. This storage completion information is updated to the server.

[0131] Step 11:

[0132] A user orders cooked food using a dedicated application. The input is the user's order information, and the output is a notification that the order information has been sent to the server. The application displays a list of available menu items.

[0133] Step 12:

[0134] The server sends the order information to the delivery person as delivery instructions. The input is the user's order information, and the output is delivery instructions to the delivery person.

[0135] Step 13:

[0136] The delivery person receives the ordered food from the food preparation facility and delivers it to the address specified by the user. The input is the delivery instruction and the cooked food, and the output is a notification to the user that the food has been delivered.

[0137] (Application example 1)

[0138] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0139] A comprehensive system is needed to effectively eliminate food waste by efficiently collecting, managing, cooking, and delivering surplus ingredients. It is also required to generate optimal menus based on ingredient information, allowing users to easily order. However, current systems lack sufficient information on ingredient freshness and generate optimal collection schedules, making efficient operation difficult.

[0140] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0141] In this invention, the server includes a means for taking a photo of surplus ingredients using a user terminal and sending the image, a generating AI model means for analyzing the sent image and registering ingredient information in a database, and a means for generating food delivery instructions based on the ingredient information and sending them to a delivery person, thereby enabling the efficient collection, management, cooking, and delivery of surplus ingredients.

[0142] A "user terminal" is a device used to take pictures of surplus ingredients and send the images. This includes smartphones.

[0143] "Image analysis" is the process of analyzing the transmitted image and extracting ingredient information.

[0144] A "generative AI model" is a system that uses machine learning algorithms to create cooking menus based on image analysis and ingredient information.

[0145] A "database" is an information system that registers and manages analyzed ingredient information.

[0146] "Food delivery" is a service that delivers food and ingredients to a location specified by the user.

[0147] "Delivery Instructions" are instructions that indicate the specific schedule and location for collecting and delivering ingredients and food.

[0148] "Cooking facilities" are facilities for cooking and storing collected food ingredients.

[0149] A "cooking menu" is a recipe or list of dishes created based on ingredient information in the database.

[0150] "User order" is an operation performed by a user to order a cooked dish.

[0151] A "smartphone application" is an app that allows users to check and order information about ingredients and menu items.

[0152] This invention is a comprehensive system for eliminating food waste through the efficient collection, management, cooking, and delivery of surplus food ingredients. The system is composed of the following elements:

[0153] 1. User Device:

[0154] Users take photos of leftover ingredients using a user device such as a smartphone. The images are then sent to a server via a dedicated application. For example, users can take photos of leftover ingredients such as tomatoes, peppers, and onions at home.

[0155] 2. Image analysis and database registration:

[0156] The server uses a generative AI model to analyze the received images. This generative AI model uses image recognition technology to determine the type, size, quantity, and freshness of ingredients. For example, it might identify a red, round object as a "tomato" and a green, elongated object as a "bell pepper." The analyzed ingredient information is then registered in a database.

[0157] 3. Ingredient collection and scheduling:

[0158] The server determines the optimal collection schedule and location based on the food ingredient information registered in the database. Collection instructions are sent to the food delivery person, informing them of the specific collection date, time, and location (e.g., the user's address). The delivery person visits the user's address at the specified date and time to collect the excess ingredients.

[0159] 4. Transportation and delivery of ingredients to cooking facilities:

[0160] The delivery staff will then transport the collected ingredients to a partner cooking facility, where they will be inspected and cleaned.

[0161] 5. Create a food menu:

[0162] The generative AI model on the server creates the optimal cooking menu based on the ingredient information registered in the database. For example, it generates a recipe for "stir-fried vegetables with tomatoes, peppers, and onions." The generated menu and recipe are sent to the cooking facility.

[0163] 6. Preparation and storage:

[0164] The kitchen staff prepares the ingredients according to the recipes created by the generative AI, and once cooking is complete, the food is stored in the appropriate way, such as refrigerated or frozen.

[0165] 7. User Orders and Delivery:

[0166] Users can order pre-cooked meals through a smartphone application. The application displays a list of available menu items, and users can select the desired dish to complete the order. The order information is sent to a server, which then sends delivery instructions to a delivery person. The delivery person collects the ordered food from the cooking facility and delivers it to the location specified by the user.

[0167] Hardware and software used:

[0168] Smartphone: Used by users to take pictures of ingredients and operate applications.

[0169] Server: Performs central processing such as image analysis, database management, and delivery instruction generation.

[0170] Generative AI models: Includes image recognition technologies such as Google® Cloud Vision API.

[0171] Cooking facilities: Facilities for cooking and storing recovered food.

[0172] Examples:

[0173] For example, a user takes a photo of two tomatoes, one green pepper, and one onion that they have left over at home and sends it to the server via the app. The generative AI model on the server analyzes the image and registers "two tomatoes, one green pepper, and one onion" in the database. A delivery person receives collection instructions, goes to the user's home to collect the ingredients, and transports them to a cooking facility. The generative AI model suggests "stir-fried vegetables," and the dish is prepared and stored at the cooking facility. Finally, the user orders the menu item via the app, and the delivery person delivers it to their home. This makes effective use of surplus ingredients and effectively eliminates food waste.

[0174] Example prompt sentence:

[0175] Image analysis prompt:

[0176] "Take a picture of leftover food in your home and analyze this image. Identify specific ingredients like tomatoes, peppers, and onions."

[0177] Recipe generation prompt:

[0178] "Generate the best recipe using the following ingredients: 2 tomatoes, 1 bell pepper, and 1 onion."

[0179] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0180] Step 1:

[0181] The user takes a photo of the surplus ingredients using a device such as a smartphone. For example, the user takes a photo of a tomato, pepper, or onion. This provides the image of the ingredients as input data.

[0182] Step 2:

[0183] The captured image is sent to the server. The user uses a dedicated application on their device to send the captured image of the ingredients to the server. The input data is the image of the ingredients, and processing begins when the server receives it.

[0184] Step 3:

[0185] The server analyzes the image. The server uses a generative AI model (such as Google Cloud Vision API) to analyze the received image. Information such as the type, size, quantity, and freshness of the ingredients is extracted from the image, and ingredient information is obtained from the input image data.

[0186] Step 4:

[0187] The ingredient information is registered in a database. The server registers the analyzed ingredient information in the database. Information such as the type and quantity of ingredients is saved in the database. The input is the analyzed ingredient information, and the output is the registration result in the database.

[0188] Step 5:

[0189] Generates an optimal collection schedule. The server determines the collection date, time, and location based on the food ingredient information registered in the database. The input data is the food ingredient information in the database, and the server generates an optimal collection schedule and notifies the delivery person as output.

[0190] Step 6:

[0191] The delivery person collects the ingredients. The delivery person follows instructions from the server and goes to the specified date, time and location to collect the ingredients. This allows the collected ingredients to be obtained from the user. The input is the server's collection instructions, and the output is the collected ingredients.

[0192] Step 7:

[0193] The collected ingredients are transported to the cooking facility. The delivery person transports the collected ingredients to the partner cooking facility. The input is the collected ingredients, and the output is the completion of transportation to the cooking facility.

[0194] Step 8:

[0195] The server generates a cooking menu. The generative AI model on the server creates the optimal cooking menu based on the ingredient information registered in the database. The input data is the ingredient information in the database, and the output is the generated cooking menu.

[0196] Step 9:

[0197] The food is cooked and stored in the kitchen. The staff at the kitchen follow the recipe generated by the server to prepare the food. After cooking, the food is stored by refrigeration or freezing. The input is the generated recipe, and the output is the cooked food.

[0198] Step 10:

[0199] The server accepts orders from users and issues delivery instructions. Users order pre-cooked food through a smartphone application, and the server issues instructions to the delivery person. The input is the user's order information, and the output is delivery instructions.

[0200] Step 11:

[0201] The delivery person delivers the ordered food to the user. The delivery person receives the food from the cooking facility and delivers it to the location specified by the user. The input is the delivery instruction and the ordered food, and the output is the completion of the food delivery.

[0202] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0203] The system of the present invention not only eliminates food waste through the efficient collection, management, cooking, and delivery of surplus ingredients, but also enables the system to suggest optimal cooking menus based on the user's emotional state. The system includes a user terminal, a server, a generation AI, a database, food delivery, cooking facilities, and an emotion engine. The detailed process and specific embodiments are described below.

[0204] 1. Take a photo of your surplus food and send it to us

[0205] Users use a smartphone or other device to take a photo of surplus food items using a dedicated application. For example, they can take a photo of leftover tomatoes, peppers, onions, or other ingredients at home. After taking the photo, they can use the application's sending function to send the image to the server.

[0206] 2. Image analysis and database registration

[0207] The server uses a generation AI to analyze the received images. This generation AI uses image recognition technology to determine the type, size, quantity, and freshness of the ingredients. For example, it identifies a red, round object as a "tomato" and a green, elongated object as a "bell pepper." The analyzed ingredient information is then registered in a database.

[0208] 3. Instructions for collection and collection of ingredients

[0209] The server determines the optimal collection schedule and location based on the food ingredient information registered in the database. Collection instructions are sent to the food delivery person, informing them of the specific collection date, time, and location (e.g., User A's address). The delivery person visits the user's address at the specified date and time to collect the excess ingredients.

[0210] 4. Transporting ingredients and delivering them to cooking facilities

[0211] The delivery staff will then transport the collected ingredients to a partner cooking facility, where they will be inspected and cleaned.

[0212] 5. Create a food menu

[0213] The AI ​​on the server creates the optimal menu based on the ingredient information registered in the database. For example, it generates a recipe for "stir-fried vegetables with tomatoes, peppers, and onions." The generated menu and recipe are then sent to the cooking facility.

[0214] 6. Cooking and storage

[0215] The kitchen staff prepares the ingredients according to the recipes created by the generative AI, and once cooking is complete, the food is stored in the appropriate way, such as refrigerated or frozen.

[0216] 7. User Sentiment Analysis

[0217] The emotion engine installed in the user device analyzes the user's emotions. For example, it uses facial recognition technology and voice analysis to recognize the user's current emotions (happiness, sadness, stress, etc.).

[0218] 8. Menu suggestions based on emotion information

[0219] The server proposes optimal meal menus based on the user's emotional information analyzed by the emotion engine. For example, if the user is feeling stressed, it will propose meal menus using ingredients that have a relaxing effect. The meal menus are also displayed on the user's device, expanding the user's options.

[0220] 9. User Orders and Delivery

[0221] Users can order pre-cooked meals through a dedicated application. The application displays a list of available menu items, allowing users to select the desired dish and complete the order. The order information is sent to a server, which then sends delivery instructions to a delivery person. The delivery person collects the ordered food from the cooking facility and delivers it to the location specified by the user.

[0222] As a specific example, a user takes a photo of two surplus tomatoes, one bell pepper, and one onion at home and sends it to the server via the app. The server's generation AI analyzes the image and registers "two tomatoes, one bell pepper, and one onion" in a database. A delivery person receives collection instructions, goes to the user's home to collect the ingredients, and transports them to a cooking facility. The generation AI suggests "stir-fried vegetables," and the dish is prepared and stored at the cooking facility. If the emotion engine on the user's device analyzes that the user is feeling stressed, the server suggests a dish with a relaxing effect. Finally, the user orders the menu item via the app, and the delivery person delivers it to their home. This makes effective use of surplus ingredients, effectively eliminating food waste, and also meets the user's emotional needs.

[0223] The processing flow will be explained below.

[0224] Step 1:

[0225] The user launches the app and takes a photo of the surplus food items, such as tomatoes, peppers, or onions, using the camera function on their smartphone.

[0226] Step 2:

[0227] The device sends the captured image to the server. When the image data is sent, it is given a timestamp and user ID information.

[0228] Step 3:

[0229] The server receives the image and analyzes it using generative AI. Specifically, it uses an image recognition algorithm to determine the type, size, quantity, and freshness of the food. For example, it identifies a red, round object as a "tomato" and evaluates its freshness based on its size and color.

[0230] Step 4:

[0231] The server registers the analysis results in a database. Along with ingredient information (e.g., "2 tomatoes, 1 bell pepper, 1 onion"), detailed information such as freshness and the date and time the photo was taken is also recorded. User emotional information is also recorded here.

[0232] Step 5:

[0233] The device activates an emotion engine to analyze the user's emotions, for example, by using voice analysis and facial recognition technology to recognize the user's current emotions (happiness, sadness, stress, etc.).

[0234] Step 6:

[0235] The server determines the optimal collection schedule and location. Based on the information in the database, it sends collection instructions to the delivery person. These instructions include the date, time, and location of the food collection (e.g., User A's address).

[0236] Step 7:

[0237] The delivery person will collect the ingredients at the specified date, time and location. The delivery person will go to User A's address and pick up the surplus ingredients.

[0238] Step 8:

[0239] The delivery person transports the collected ingredients to the cooking facility. During the transport, the condition of the ingredients is easily checked and any problems are reported to the server.

[0240] Step 9:

[0241] The server communicates ingredient information to the cooking facility, including the type, quantity, and freshness of the ingredients.

[0242] Step 10:

[0243] The generative AI creates the optimal cooking menu. Based on the ingredient information in the database and the user's emotional information, it generates a recipe such as "Stir-fried vegetables with tomatoes, peppers, and onions" and sends it to the cooking facility.

[0244] Step 11:

[0245] The staff at the kitchen will prepare the food according to the recipes generated by the AI, and once the food is cooked, it will be stored appropriately, for example by refrigerating or freezing it.

[0246] Step 12:

[0247] Users order pre-cooked meals through a dedicated app, which displays available menu items, and users can select the desired dish to complete the order.

[0248] Step 13:

[0249] The server sends delivery instructions to the delivery person based on the order information. The delivery person is instructed to "Deliver the food from the cooking facility to User B's address."

[0250] Step 14:

[0251] The delivery person picks up the ordered food at the cooking facility and delivers it to the address of User B. After confirming that the food has been delivered safely, the delivery is reported to the server as completed.

[0252] As a specific example, a user takes a photo of two surplus tomatoes, one bell pepper, and one onion at home and sends it to a server via the app. The server's generation AI analyzes the image and registers "two tomatoes, one bell pepper, and one onion" in a database. The device's emotion engine analyzes the user's stress and registers this information in the database. A delivery person receives collection instructions, goes to the user's home, collects the ingredients, and transports them to a cooking facility. The generation AI suggests a "relaxing stir-fry vegetable dish" based on the user's emotions, and the dish is prepared and stored at the cooking facility. Finally, the user orders the menu item via the app, and the delivery person delivers it to their home. This makes effective use of surplus ingredients, effectively eliminating food waste, and also meets the user's emotional needs.

[0253] Example 2

[0254] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0255] In modern society, excessive food purchases and improper storage result in large amounts of surplus food, making food waste a serious problem. Furthermore, since it is difficult to provide meals that meet the diverse emotional needs of users, appropriate measures are required to improve user satisfaction. To solve these issues, an efficient and optimal system is required.

[0256] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0257] In this invention, the server includes a user terminal means for taking pictures of surplus ingredients, a generating AI means for analyzing the transmitted images and registering ingredient information in a database, and a means for proposing optimal recipes based on emotional information, thereby enabling effective use of surplus ingredients and proposing menus that meet the emotional needs of users.

[0258] "Surplus food" refers to food that is left unused in homes, restaurants, etc.

[0259] "User terminal" refers to a portable electronic device such as a smartphone or tablet used by a user.

[0260] "Generative AI" refers to artificial intelligence that uses machine learning algorithms to analyze data and perform specific tasks automatically.

[0261] A "database" refers to a system for efficiently storing, managing, and retrieving data.

[0262] "Food delivery" refers to a service that delivers pre-cooked food to the customer's home.

[0263] "Delivery person" refers to a person whose job is to deliver goods or food from a designated location to another.

[0264] "Cooking facility" means the place where food is prepared and stored.

[0265] "Emotion analysis" refers to the process of measuring and analyzing a user's emotional state using data such as facial expressions and voice.

[0266] "Cooking Menu" refers to a recipe or list of dishes prepared using specific ingredients.

[0267] "Proposal means" refers to techniques and methods that present optimal options based on the user's requests and circumstances.

[0268] This invention is a system that efficiently collects, analyzes, manages, cooks, and delivers surplus ingredients. It also has a function that suggests optimal cooking menus based on the user's emotional state. This system consists of a user terminal, a server, a generation AI, a database, food delivery, cooking facilities, and an emotion engine.

[0269] System configuration

[0270] 1. User Device

[0271] The user takes a photo of the surplus ingredients using a user device such as a smartphone or tablet, launches a dedicated application, takes a photo of the ingredients, and sends it to the server.

[0272] 2. Server

[0273] The server uses a generative AI model to analyze the received images. This generative AI model uses image recognition technology to determine the type, quantity, and freshness of ingredients. The analysis results are then registered in a database.

[0274] 3. Generative AI Models

[0275] The generative AI creates optimal recipes based on the collected information on surplus ingredients. For example, it can generate a recipe for "stir-fried vegetables with tomatoes, peppers, and onions."

[0276] 4. Database

[0277] The database stores and manages the analyzed ingredient information, recording information such as the type, quantity, and freshness of ingredients.

[0278] 5. Food delivery

[0279] Based on the food ingredient information in the database, the server instructs the food delivery service's delivery staff on the optimal collection schedule and location. The delivery staff follows these instructions, collects the surplus ingredients at the specified date, time, and location, and delivers them to a partner cooking facility.

[0280] 6. Cooking facilities

[0281] The food preparation facility receives the collected ingredients, checks and cleans them, and then prepares them according to the recipes provided by the AI. The cooked food is then stored in an appropriate manner, such as refrigerated or frozen.

[0282] 7. Emotion Engine

[0283] The emotion engine installed on the user's device uses facial recognition technology and voice analysis to analyze the user's current emotional state. The analysis results are sent to a server, which then suggests optimal cooking menus based on the user's emotions.

[0284] Examples of concrete examples and prompts

[0285] As a specific example, a user takes a photo of two leftover tomatoes, one green pepper, and one onion at home and sends it to a server via a dedicated app. The server's generation AI analyzes the image and registers "two tomatoes, one green pepper, and one onion" in a database. A delivery person receives collection instructions, goes to the user's home to collect the ingredients, and transports them to a cooking facility. The generation AI generates a recipe for "stir-fried vegetables," which is then cooked at the cooking facility. Furthermore, the emotion engine on the user's device analyzes that the user is in a stressful state, and the server suggests a dish menu with a relaxing effect. The user then orders the suggested menu using a dedicated app, and a delivery person delivers the food from the cooking facility to the user's home.

[0286] An example of a prompt is as follows:

[0287] "I have two tomatoes, one bell pepper, and one onion. Can you suggest a dish I can make using these ingredients?"

[0288] "Please suggest a meal plan when the user is feeling stressed."

[0289] This allows for effective use of surplus ingredients and menu suggestions that meet the emotional needs of users.

[0290] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0291] Step 1:

[0292] Input: Images of surplus food taken by a user using a user device such as a smartphone.

[0293] Specific operation: The user launches a dedicated application and takes a photo of surplus ingredients (e.g., two tomatoes, one green pepper, and one onion).

[0294] Output: The captured image is saved in the user's device.

[0295] Next process: The saved image is sent to the server.

[0296] Step 2:

[0297] Input: Ingredient images sent from the user device.

[0298] Specific operation: The user taps the send button in the application to send the captured image to the server.

[0299] Output: The image data arrives at the server.

[0300] Next process: The server analyzes the received image.

[0301] Step 3:

[0302] Input: Image data received by the server.

[0303] Specific operation: The server inputs the received image into the generative AI model and uses image recognition technology to analyze the type, quantity, and freshness of ingredients. For example, the generative AI recognizes a red, round shape as a "tomato."

[0304] Output: The analysis result is "2 tomatoes, 1 bell pepper, and 1 onion."

[0305] Next process: The analysis results are registered in a database.

[0306] Step 4:

[0307] Input: Parsed ingredient information.

[0308] Specific operation: The server registers the analysis results in a database.

[0309] Output: Stored in the database as "2 tomatoes, 1 bell pepper, 1 onion".

[0310] Next action: Collection instructions are generated from the database.

[0311] Step 5:

[0312] Input: Ingredient information stored in the database.

[0313] Specific operation: Based on the ingredient information in the database, the server instructs the food delivery person on the optimal collection schedule and location. For example, it may instruct the person to "collect two tomatoes, one green pepper, and one onion from User A's address."

[0314] Output: Collection instructions received by the courier.

[0315] Next step: The delivery person collects the ingredients.

[0316] Step 6:

[0317] Input: Collection instructions received by the courier.

[0318] Specific operation: The delivery person will go to the user's address at the specified date and time and collect the excess ingredients.

[0319] Output: Recovered ingredients.

[0320] Next process: The recovered ingredients are transported to a food preparation facility.

[0321] Step 7:

[0322] Input: Recovered ingredients.

[0323] Specific operation: The delivery person transports the collected ingredients to a partner cooking facility.

[0324] Output: Food arriving at the cooking facility.

[0325] Next step: the ingredients are cooked.

[0326] Step 8:

[0327] Input: Ingredients arriving at the cooking facility.

[0328] Specific actions: The kitchen staff inspects the collected ingredients, cleans them, and then cooks them according to the recipe provided by the generative AI. For example, they make a vegetable stir-fry using tomatoes, peppers, and onions.

[0329] Output: Cooked food.

[0330] Next process: The dish is saved.

[0331] Step 9:

[0332] Input: Cooked food.

[0333] Specific Behavior: Store cooked food in the correct manner, such as by refrigerating or freezing.

[0334] Output: Saved dishes.

[0335] Next process: Emotion analysis on the user device.

[0336] Step 10:

[0337] Input: The user's emotional state.

[0338] Specific operation: The emotion engine installed on the user's device analyzes the user's emotional state using facial recognition and voice analysis.

[0339] Output: Parsed emotion information.

[0340] Next process: Menu suggestions are made based on emotional information.

[0341] Step 11:

[0342] Input: Parsed emotion information.

[0343] Specific operation: The server proposes the most suitable food menu based on the emotional information analyzed by the emotion engine. For example, if the user is feeling stressed, it will propose dishes that have a relaxing effect.

[0344] Output: A menu of dishes suggested to the user.

[0345] Next: The user orders food.

[0346] Step 12:

[0347] Input: Suggested food menu.

[0348] Specific operation: The user uses a dedicated application to select the desired dish from the menu and complete the order.

[0349] Output: Order information.

[0350] Next step: The ordered food is delivered.

[0351] Step 13:

[0352] Input: User's order information.

[0353] Specific operation: The server receives the order information and sends delivery instructions to the delivery person. The delivery person picks up the ordered food from the cooking facility and delivers it to the location specified by the user.

[0354] Output: The food delivered to the user.

[0355] Next process: The user's meal is completed and the system process ends.

[0356] (Application example 2)

[0357] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0358] This invention aims to address the issue of food waste while improving users' life satisfaction by suggesting optimal cooking menus based on the user's emotional state. Conventional food delivery services have had difficulty making effective use of surplus ingredients or providing personalized services based on the user's emotional state. This has led to increased food waste and a poor user experience.

[0359] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a generation AI means that analyzes the transmitted image and registers the ingredient information in a database, an emotion engine means that analyzes the user's emotional state using the user terminal, and a server means that suggests a dish menu based on the user's emotional state. This makes it possible to effectively use surplus ingredients and suggest a dish menu that is optimal for the user's current emotional state.

[0360] "Surplus food" refers to food that is left over in homes or facilities and that may be discarded without being used.

[0361] A "user terminal" is a device that a user uses to take pictures of surplus ingredients and input order information, and includes smartphones, tablets, and the like.

[0362] The "generative AI means" is an artificial intelligence technology that analyzes the transmitted image, identifies ingredient information, and registers it in a database.

[0363] The "emotion engine means" is a technology for analyzing the user's emotional state, and includes facial recognition technology, voice analysis technology, and the like.

[0364] The "server means" is a central processing unit that issues collection instructions and suggests cooking menus based on ingredient information and the user's emotional state.

[0365] "Delivery personnel means" refers to personnel and vehicles that actually collect surplus ingredients and deliver cooked meals to users.

[0366] "Affiliated Food Facility" means a food facility or restaurant with which we have contracted to prepare and store recovered food.

[0367] The "means for suggesting a cooking menu" is a technology that allows the generative AI to create the optimal cooking menu based on the user's emotional state and ingredient information in the database.

[0368] A "database" is an electronic data storage device for centrally managing analyzed ingredient information, user order information, and the like.

[0369] Overall system overview

[0370] This invention is a system that efficiently collects and manages surplus ingredients, and proposes and delivers meal plans tailored to the user's emotional state. The following describes in detail how this system is implemented.

[0371] Hardware and Software Use

[0372] This system mainly uses the following hardware and software:

[0373] Smartphone: Used to take photos of surplus ingredients and analyze user emotions

[0374] Server: Used for image analysis, food ingredient information registration using generative AI, user emotional state analysis, and menu suggestions

[0375] Generative AI model: Analyzes ingredient information and generates optimal cooking menus

[0376] Emotion engine: Analyzes the user's emotional state through facial recognition and voice analysis

[0377] Database: Stores information about ingredients and user orders

[0378] Explanation of program processing

[0379] 1. Photograph and send your surplus ingredients:

[0380] Users use a dedicated smartphone app to take a photo of leftover ingredients at home, and the app sends the image to a server.

[0381] 2. Image analysis and database registration:

[0382] The server uses a generative AI model to analyze the received images, identifying the type, quantity, and freshness of ingredients from the images and registering this information in a database.

[0383] 3. Instructions for collection and collection of ingredients:

[0384] The server generates an optimal collection schedule based on the food ingredient information registered in the database and sends instructions to the delivery staff, who then collect the ingredients at the specified date and time and transport them to the partner cooking facility.

[0385] 4. Create a food menu:

[0386] The generative AI model on the server creates the optimal recipe based on the ingredient information registered in the database. For example, it generates a recipe for "stir-fried vegetables with tomatoes, peppers, and onions."

[0387] 5. Cooking and storage:

[0388] Staff at partner kitchens will prepare the ingredients according to the recipes created by the generative AI, and the cooked food will be stored in an appropriate manner, such as refrigerated or frozen.

[0389] 6. User sentiment analysis:

[0390] The emotion engine on the user device analyzes the user's emotional state using a camera and microphone, and the analyzed emotion information is sent to the server.

[0391] 7. Menu suggestions based on emotion information:

[0392] The server uses a generative AI model to suggest optimal meal plans based on the analyzed emotional information. For example, if the user is feeling stressed, the server suggests meal plans with a relaxing effect.

[0393] 8. User Orders and Delivery:

[0394] The user places an order by selecting from the menu of suggested dishes through a dedicated app. The server receives the order information and sends delivery instructions to the delivery person. The delivery person then picks up the food from the cooking facility and delivers it to the user.

[0395] Specific examples

[0396] Specifically, User A takes a photo of two tomatoes, one green pepper, and one onion that they have left over at home and sends it to the server via the app. The server's generation AI analyzes this and registers "two tomatoes, one green pepper, and one onion" in a database. A delivery person then receives collection instructions, goes to User A's home to collect the ingredients, and transports them to a cooking facility. The generation AI suggests "stir-fried vegetables," and the dish is prepared at the cooking facility. When the emotion engine on the user's device analyzes that User A is feeling stressed, the server suggests "stir-fried vegetables with sesame flavor" as a dish that has a relaxing effect. Finally, User A orders this menu through the app, and a delivery person delivers it to their home. This makes it possible to make effective use of surplus ingredients, reduce food waste, and provide a service that is tailored to the user's emotions.

[0397] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0398] Step 1:

[0399] When a user realizes that they have surplus ingredients at home, they take a photo of the ingredients using a dedicated smartphone app. The input is the photo of the ingredients, and the output is image data stored on the smartphone. The smartphone stores this image data within the application.

[0400] Step 2:

[0401] A smartphone application sends a captured image to a server. The input is image data, and the output is the sent image data. Specifically, the application uploads the image file to the server using an HTTP request.

[0402] Step 3:

[0403] The server uses a generative AI model to analyze the transmitted image. The input is image data, and the output is analyzed ingredient information (type, quantity, freshness, etc.). The generative AI model analyzes the data using image recognition technology to identify ingredient information.

[0404] Step 4:

[0405] The server registers the analyzed ingredient information in a database. The input is the analyzed ingredient information, and the output is a record to be saved in the database. The server inserts the ingredient information into the database using an SQL query.

[0406] Step 5:

[0407] The server generates an optimal collection schedule based on the food ingredient information registered in the database and sends collection instructions to the delivery staff. The input is the food ingredient information in the database, and the output is the collection instructions sent to the delivery staff. The server uses a scheduling algorithm to determine the optimal collection date, time, and location.

[0408] Step 6:

[0409] The delivery person visits the user's address at the specified date and time to collect the excess ingredients. The input is the collection instruction from the server, and the output is the collected ingredients. The delivery person safely transports the ingredients using a container specially designed for food.

[0410] Step 7:

[0411] The delivery staff transports the collected ingredients to the partner cooking facility. The input is the collected ingredients, and the output is the ingredients delivered to the cooking facility. The delivery staff uses appropriate transportation methods to ensure the ingredients are kept fresh.

[0412] Step 8:

[0413] The generative AI model on the server creates the optimal cooking menu based on the ingredient information registered in the database. The input is the ingredient information in the database, and the output is the generated cooking menu. The generative AI model generates the optimal recipe by taking into account the combination of ingredients and cooking methods.

[0414] Step 9:

[0415] Partnered kitchens prepare ingredients according to recipes created by the generative AI. The input is the generated menu and ingredients, and the output is the cooked food. Kitchen staff follow the set procedures to create high-quality dishes.

[0416] Step 10:

[0417] The emotion engine on the user device analyzes the user's emotional state. The input is the user's facial image and voice data, and the output is analyzed emotional information. The emotion engine recognizes the user's emotions using facial recognition technology and voice analysis technology.

[0418] Step 11:

[0419] The server uses a generative AI model to propose optimal cooking menus based on the analyzed emotional information. The input is emotional information and ingredient information, and the output is an optimal cooking menu based on the emotions. The server proposes menus that have a relaxing effect or that are invigorating depending on the emotions.

[0420] Step 12:

[0421] Users place their orders by selecting from the proposed menu through a dedicated app. The input is the menu information, and the output is the user's order data. The app provides an interface that allows users to easily select menu items and complete their orders.

[0422] Step 13:

[0423] The server receives the user's order information and sends delivery instructions to the delivery person. The input is the user's order information and the output is delivery instructions. The server generates the optimal delivery route and schedule and sends the delivery instructions to the delivery person.

[0424] Step 14:

[0425] The delivery person receives the food from the cooking facility and delivers it to the location specified by the user. The input is the cooked food and delivery instructions, and the output is the food delivered to the user. The delivery person is required to deliver the ordered food safely and quickly.

[0426] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0427] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0428] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0429] [Second embodiment]

[0430] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0431] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0432] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0433] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0434] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0435] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0436] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0437] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0438] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0439] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0440] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0441] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0442] The system of the present invention provides a comprehensive solution to eliminate food waste through the efficient collection, management, cooking, and delivery of surplus food ingredients. The detailed process and specific embodiments are described below.

[0443] 1. Take a photo of your surplus food and send it to us

[0444] Users use a smartphone or other device to take a photo of surplus food items using a dedicated application. For example, they can take a photo of leftover tomatoes, peppers, onions, or other ingredients at home. After taking the photo, they can use the application's sending function to send the image to the server.

[0445] 2. Image analysis and database registration

[0446] The server uses a generation AI to analyze the received images. This generation AI uses image recognition technology to determine the type, size, quantity, and freshness of the ingredients. For example, it identifies a red, round object as a "tomato" and a green, elongated object as a "bell pepper." The analyzed ingredient information is then registered in a database.

[0447] 3. Instructions for collection and collection of ingredients

[0448] The server determines the optimal collection schedule and location based on the food ingredient information registered in the database. Collection instructions are sent to the food delivery person, informing them of the specific collection date, time, and location (e.g., User A's address). The delivery person visits the user's address at the specified date and time to collect the excess ingredients.

[0449] 4. Transporting ingredients and delivering them to cooking facilities

[0450] The delivery staff will then transport the collected ingredients to a partner cooking facility, where they will be inspected and cleaned.

[0451] 5. Create a food menu

[0452] The AI ​​on the server creates the optimal menu based on the ingredient information registered in the database. For example, it generates a recipe for "stir-fried vegetables with tomatoes, peppers, and onions." The generated menu and recipe are then sent to the cooking facility.

[0453] 6. Cooking and storage

[0454] The kitchen staff prepares the ingredients according to the recipes created by the generative AI, and once cooking is complete, the food is stored in the appropriate way, such as refrigerated or frozen.

[0455] 7. User Orders and Delivery

[0456] Users can order pre-cooked meals through a dedicated application. The application displays a list of available menu items, allowing users to select the desired dish and complete the order. The order information is sent to a server, which then sends delivery instructions to a delivery person. The delivery person collects the ordered food from the cooking facility and delivers it to the location specified by the user.

[0457] As a specific example, a user takes a photo of two tomatoes, one green pepper, and one onion that they have left over at home and sends it to the server via the app. The server's generation AI analyzes the photo and registers "two tomatoes, one green pepper, and one onion" in the database. A delivery person receives collection instructions, goes to the user's home to collect the ingredients, and transports them to a cooking facility. The generation AI suggests "stir-fried vegetables," and the dish is prepared and stored at the cooking facility. Finally, the user orders the menu item via the app, and the delivery person delivers it to their home. This makes effective use of surplus ingredients and effectively eliminates food waste.

[0458] The processing flow will be explained below.

[0459] Step 1:

[0460] The user launches the app and takes a photo of the surplus food items, such as tomatoes, peppers, or onions, using the camera function on their smartphone.

[0461] Step 2:

[0462] The device sends the captured image to the server. When the image data is sent, it is given a timestamp and user ID information.

[0463] Step 3:

[0464] The server receives the image and analyzes it using generative AI. Specifically, it uses an image recognition algorithm to determine the type, size, quantity, and freshness of the food. For example, it identifies a red, round object as a "tomato" and evaluates its freshness based on its size and color.

[0465] Step 4:

[0466] The server registers the analysis results in a database, recording detailed information such as the freshness of the ingredients (e.g., "2 tomatoes, 1 bell pepper, 1 onion") and the date and time the photo was taken.

[0467] Step 5:

[0468] The server determines the optimal collection schedule and location. Based on the information in the database, it sends collection instructions to the delivery person. These instructions include the date, time, and location of the food collection (e.g., User A's address).

[0469] Step 6:

[0470] The delivery person will collect the ingredients at the specified date, time and location. The delivery person will go to User A's address and pick up the surplus ingredients.

[0471] Step 7:

[0472] The delivery person transports the collected ingredients to the cooking facility. During the transport, the condition of the ingredients is easily checked and any problems are reported to the server.

[0473] Step 8:

[0474] The server communicates ingredient information to the cooking facility, including the type, quantity, and freshness of the ingredients.

[0475] Step 9:

[0476] The generative AI creates the optimal menu, generating recipes such as "stir-fried vegetables with tomatoes, peppers, and onions" based on the ingredients in the database, and sending them to the cooking facility.

[0477] Step 10:

[0478] The staff at the kitchen will prepare the food according to the recipes generated by the AI, and once the food is cooked, it will be stored appropriately, for example by refrigerating or freezing it.

[0479] Step 11:

[0480] Users order pre-cooked meals through a dedicated app, which displays available menu items, and users can select the desired dish to complete the order.

[0481] Step 12:

[0482] The server sends delivery instructions to the delivery person based on the order information. The delivery person is instructed to "Deliver the food from the cooking facility to User B's address."

[0483] Step 13:

[0484] The delivery person picks up the ordered food at the cooking facility and delivers it to the address of User B. After confirming that the food has been delivered safely, the delivery is reported to the server as completed.

[0485] Example 1

[0486] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0487] Currently, the effective use of surplus ingredients and the problem of food waste are social issues, but efficient systems for collection, management, cooking, and delivery are not fully established. There is also a lack of advanced analytical technology to determine the freshness of ingredients and propose appropriate cooking menus. For this reason, a system that can effectively utilize surplus ingredients is needed.

[0488] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0489] In this invention, the server includes a user terminal means for taking pictures of surplus ingredients, a means for sending the taken images to the server, and a generating AI means on the server for analyzing the sent images and registering ingredient information in a database, thereby enabling efficient collection, analysis, and management of surplus ingredients.

[0490] The server also includes a means for sending a collection instruction to a delivery person based on the ingredient information, and a delivery person means for collecting the ingredients according to the collection instruction and transporting them to the cooking facility, thereby enabling the ingredients to be collected and delivered to the cooking facility quickly.

[0491] Furthermore, the server includes a generation AI means for creating an optimal menu based on the ingredient information in the database, a means for cooking and storing ingredients at a cooking facility, a delivery staff means for delivering cooked food based on the user's order, a means for reporting the status of the food en route to the cooking facility to the server in real time, and a means for updating the server with information on completion of storage, thereby enabling the effective use of ingredients, appropriate cooking and storage, and prompt delivery to the user.

[0492] "Surplus ingredients" refers to ingredients or food that are left over and not consumed.

[0493] "User terminal means" refers to a device used by a user, such as a computer or smartphone, that has the function of photographing ingredients and transmitting information about them.

[0494] "Server" refers to a computer system that processes, stores, and transmits data over a network.

[0495] "Generative AI means" refers to technologies that use artificial intelligence (AI) to analyze data and generate specific functions or results.

[0496] "Delivery means" refers to the person or system that collects ingredients or food from a designated location and transports them to a designated location.

[0497] "Ingredient information" refers to data including details such as the type, quantity, and freshness of ingredients.

[0498] A "database" is a data collection point and a system for organizing, storing, and searching various information.

[0499] "Cooking facility" refers to a facility that receives, cooks, and preserves food ingredients.

[0500] "Food Menu" means a list of the types of food available and their details.

[0501] "Freshness" refers to the degree of freshness of food or ingredients, and is usually a standard for evaluating quality, expiration dates, etc.

[0502] "Preservation measures" refers to measures such as refrigeration, freezing, and packaging that are used to store cooked food and ingredients in an appropriate state.

[0503] The system of the present invention provides a comprehensive solution to eliminate food waste through the efficient collection, management, cooking, and delivery of surplus ingredients. Specific embodiments of the system of the present invention are described below.

[0504] Using a smartphone or other user device, a user can take a photo of surplus food items using a dedicated application. For example, they can take a photo of leftover tomatoes, peppers, onions, or other ingredients at home, and press the send button on the application to send the image to the server.

[0505] The server then launches a generative AI model to analyze the received images. This generative AI model uses image recognition technologies such as TensorFlow and PyTorch to determine the type, size, quantity, and freshness of the ingredients and registers them in a database. For example, it might identify a red, round object as a "tomato" and a green, elongated object as a "bell pepper."

[0506] Based on the food ingredient information registered in the database, the server determines the optimal collection schedule and location. Collection instructions are sent to the delivery person's device, and the delivery person visits the user's address at the specified date and time to collect the excess food. The GPS-equipped device provides the delivery person with the optimal route, supporting efficient collection.

[0507] The delivery person transports the collected ingredients to the cooking facility. During this process, the status of the delivery is reported to the server in real time. Upon arrival at the cooking facility, information confirming the arrival of the ingredients is sent to the server. The cooking facility staff checks the ingredients and cleans up if necessary.

[0508] The generative AI model on the server generates the optimal dish menu based on the ingredient information registered in the database. This process uses a natural language generation model (e.g., GPT-4). For example, a recipe for "stir-fried vegetables with tomatoes, peppers, and onions" is generated and sent to the cooking facility.

[0509] The staff at the kitchen prepares the ingredients according to the recipe created by the generative AI. Once the food is cooked, it is stored in an appropriate manner, such as refrigerated or frozen. This information about the completion of storage is updated to the server.

[0510] Users can order pre-cooked meals through a dedicated application. The application displays a list of available menu items, and users can select the desired dish to complete the order. The order information is sent to a server, which then sends delivery instructions to a delivery person. The delivery person collects the ordered food from the cooking facility and delivers it to the address specified by the user.

[0511] Examples:

[0512] The user takes a photo of two surplus tomatoes, one bell pepper, and one onion at home and sends it to the server via the app. The server's generation AI analyzes the photo and registers it in the database as "two tomatoes, one bell pepper, and one onion." A delivery person receives collection instructions, goes to the user's home to collect the ingredients, and transports them to a cooking facility. The generation AI suggests a recipe for "stir-fried vegetables," and the dish is prepared at the cooking facility based on this recipe and stored. Finally, the user orders the menu item via the app, and the delivery person delivers it to their home. This makes effective use of surplus ingredients and effectively eliminates food waste.

[0513] Example prompt for a generative AI model:

[0514] "Please suggest the best recipe based on the ingredient information received from the user. For example, if two tomatoes, one bell pepper, and one onion are provided, please consider what kind of dish can be made using them and provide a detailed recipe."

[0515] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0516] Step 1:

[0517] The user uses a dedicated smartphone application to take a photo of surplus ingredients. For example, they can take a photo of leftover tomatoes, peppers, onions, and other ingredients at home. The input is an image of each ingredient, and the output is an image file.

[0518] Step 2:

[0519] The user presses the send button in the dedicated application to send the captured image to the server. The input is the image file obtained in step 1, and the output is a notification that the image file has been uploaded to the server.

[0520] Step 3:

[0521] The server launches a generative AI model to analyze the received image. The input is the transmitted image file, and the generative AI model performs image recognition using TensorFlow or PyTorch. The output is the recognized ingredient information (e.g., 2 tomatoes, 1 bell pepper, 1 onion). The server registers this information in a database.

[0522] Step 4:

[0523] The server determines the optimal collection schedule and location based on the food ingredient information registered in the database. The input is the food ingredient information registered in the database, and the output is collection instructions to the delivery person. The collection instructions include the specific collection date, time, and location.

[0524] Step 5:

[0525] The delivery person visits the user's address at the specified date and time to collect the excess ingredients. The input is the collection instruction from the server, and the output is the collected ingredients. The delivery person's device is equipped with a GPS, which provides the optimal route.

[0526] Step 6:

[0527] The delivery person transports the collected ingredients to the cooking facility and reports the status of the delivery to the server in real time. The input is the collected ingredients, and the output is a notification of arrival at the cooking facility.

[0528] Step 7:

[0529] The kitchen staff checks the food as it arrives and cleans it up as necessary. The input is the delivered food, and the output is the cleaned food.

[0530] Step 8:

[0531] A generative AI model on the server generates optimal cooking menus based on ingredient information registered in a database. The input is ingredient information from the database, and the generative AI model uses natural language generation technologies such as GPT-4. The output is the generated cooking menu and recipe.

[0532] Step 9:

[0533] The kitchen staff prepares the ingredients according to the recipes created by the generative AI model. The input is the generated food menu and recipe, and the output is the cooked food.

[0534] Step 10:

[0535] Once the food is cooked, it is stored in an appropriate way, such as refrigerated or frozen. The input is the cooked food, and the output is the stored food. This storage completion information is updated to the server.

[0536] Step 11:

[0537] A user orders cooked food using a dedicated application. The input is the user's order information, and the output is a notification that the order information has been sent to the server. The application displays a list of available menu items.

[0538] Step 12:

[0539] The server sends the order information to the delivery person as delivery instructions. The input is the user's order information, and the output is delivery instructions to the delivery person.

[0540] Step 13:

[0541] The delivery person receives the ordered food from the food preparation facility and delivers it to the address specified by the user. The input is the delivery instruction and the cooked food, and the output is a notification to the user that the food has been delivered.

[0542] (Application example 1)

[0543] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0544] A comprehensive system is needed to effectively eliminate food waste by efficiently collecting, managing, cooking, and delivering surplus ingredients. It is also required to generate optimal menus based on ingredient information, allowing users to easily order. However, current systems lack sufficient information on ingredient freshness and generate optimal collection schedules, making efficient operation difficult.

[0545] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0546] In this invention, the server includes a means for taking a photo of surplus ingredients using a user terminal and sending the image, a generating AI model means for analyzing the sent image and registering ingredient information in a database, and a means for generating food delivery instructions based on the ingredient information and sending them to a delivery person, thereby enabling the efficient collection, management, cooking, and delivery of surplus ingredients.

[0547] A "user terminal" is a device used to take pictures of surplus ingredients and send the images. This includes smartphones.

[0548] "Image analysis" is the process of analyzing the transmitted image and extracting ingredient information.

[0549] A "generative AI model" is a system that uses machine learning algorithms to create cooking menus based on image analysis and ingredient information.

[0550] A "database" is an information system that registers and manages analyzed ingredient information.

[0551] "Food delivery" is a service that delivers food and ingredients to a location specified by the user.

[0552] "Delivery Instructions" are instructions that indicate the specific schedule and location for collecting and delivering ingredients and food.

[0553] "Cooking facilities" are facilities for cooking and storing collected food ingredients.

[0554] A "cooking menu" is a recipe or list of dishes created based on ingredient information in the database.

[0555] "User order" is an operation performed by a user to order a cooked dish.

[0556] A "smartphone application" is an app that allows users to check and order information about ingredients and menu items.

[0557] This invention is a comprehensive system for eliminating food waste through the efficient collection, management, cooking, and delivery of surplus food ingredients. The system is composed of the following elements:

[0558] 1. User Device:

[0559] Users take photos of leftover ingredients using a user device such as a smartphone. The images are then sent to a server via a dedicated application. For example, users can take photos of leftover ingredients such as tomatoes, peppers, and onions at home.

[0560] 2. Image analysis and database registration:

[0561] The server uses a generative AI model to analyze the received images. This generative AI model uses image recognition technology to determine the type, size, quantity, and freshness of ingredients. For example, it might identify a red, round object as a "tomato" and a green, elongated object as a "bell pepper." The analyzed ingredient information is then registered in a database.

[0562] 3. Ingredient collection and scheduling:

[0563] The server determines the optimal collection schedule and location based on the food ingredient information registered in the database. Collection instructions are sent to the food delivery person, informing them of the specific collection date, time, and location (e.g., the user's address). The delivery person visits the user's address at the specified date and time to collect the excess ingredients.

[0564] 4. Transportation and delivery of ingredients to cooking facilities:

[0565] The delivery staff will then transport the collected ingredients to a partner cooking facility, where they will be inspected and cleaned.

[0566] 5. Create a food menu:

[0567] The generative AI model on the server creates the optimal cooking menu based on the ingredient information registered in the database. For example, it generates a recipe for "stir-fried vegetables with tomatoes, peppers, and onions." The generated menu and recipe are sent to the cooking facility.

[0568] 6. Preparation and storage:

[0569] The kitchen staff prepares the ingredients according to the recipes created by the generative AI, and once cooking is complete, the food is stored in the appropriate way, such as refrigerated or frozen.

[0570] 7. User Orders and Delivery:

[0571] Users can order pre-cooked meals through a smartphone application. The application displays a list of available menu items, and users can select the desired dish to complete the order. The order information is sent to a server, which then sends delivery instructions to a delivery person. The delivery person collects the ordered food from the cooking facility and delivers it to the location specified by the user.

[0572] Hardware and software used:

[0573] Smartphone: Used by users to take pictures of ingredients and operate applications.

[0574] Server: Performs central processing such as image analysis, database management, and delivery instruction generation.

[0575] Generative AI models: Includes image recognition technologies such as Google Cloud Vision API.

[0576] Cooking facilities: Facilities for cooking and storing recovered food.

[0577] Examples:

[0578] For example, a user takes a photo of two tomatoes, one green pepper, and one onion that they have left over at home and sends it to the server via the app. The generative AI model on the server analyzes the image and registers "two tomatoes, one green pepper, and one onion" in the database. A delivery person receives collection instructions, goes to the user's home to collect the ingredients, and transports them to a cooking facility. The generative AI model suggests "stir-fried vegetables," and the dish is prepared and stored at the cooking facility. Finally, the user orders the menu item via the app, and the delivery person delivers it to their home. This makes effective use of surplus ingredients and effectively eliminates food waste.

[0579] Example prompt sentence:

[0580] Image analysis prompt:

[0581] "Take a picture of leftover food in your home and analyze this image. Identify specific ingredients like tomatoes, peppers, and onions."

[0582] Recipe generation prompt:

[0583] "Generate the best recipe using the following ingredients: 2 tomatoes, 1 bell pepper, and 1 onion."

[0584] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0585] Step 1:

[0586] The user takes a photo of the surplus ingredients using a device such as a smartphone. For example, the user takes a photo of a tomato, pepper, or onion. This provides the image of the ingredients as input data.

[0587] Step 2:

[0588] The captured image is sent to the server. The user uses a dedicated application on their device to send the captured image of the ingredients to the server. The input data is the image of the ingredients, and processing begins when the server receives it.

[0589] Step 3:

[0590] The server analyzes the image. The server uses a generative AI model (such as Google Cloud Vision API) to analyze the received image. Information such as the type, size, quantity, and freshness of the ingredients is extracted from the image, and ingredient information is obtained from the input image data.

[0591] Step 4:

[0592] The ingredient information is registered in a database. The server registers the analyzed ingredient information in the database. Information such as the type and quantity of ingredients is saved in the database. The input is the analyzed ingredient information, and the output is the registration result in the database.

[0593] Step 5:

[0594] Generates an optimal collection schedule. The server determines the collection date, time, and location based on the food ingredient information registered in the database. The input data is the food ingredient information in the database, and the server generates an optimal collection schedule and notifies the delivery person as output.

[0595] Step 6:

[0596] The delivery person collects the ingredients. The delivery person follows instructions from the server and goes to the specified date, time and location to collect the ingredients. This allows the collected ingredients to be obtained from the user. The input is the server's collection instructions, and the output is the collected ingredients.

[0597] Step 7:

[0598] The collected ingredients are transported to the cooking facility. The delivery person transports the collected ingredients to the partner cooking facility. The input is the collected ingredients, and the output is the completion of transportation to the cooking facility.

[0599] Step 8:

[0600] The server generates a cooking menu. The generative AI model on the server creates the optimal cooking menu based on the ingredient information registered in the database. The input data is the ingredient information in the database, and the output is the generated cooking menu.

[0601] Step 9:

[0602] The food is cooked and stored in the kitchen. The staff at the kitchen follow the recipe generated by the server to prepare the food. After cooking, the food is stored by refrigeration or freezing. The input is the generated recipe, and the output is the cooked food.

[0603] Step 10:

[0604] The server accepts orders from users and issues delivery instructions. Users order pre-cooked food through a smartphone application, and the server issues instructions to the delivery person. The input is the user's order information, and the output is delivery instructions.

[0605] Step 11:

[0606] The delivery person delivers the ordered food to the user. The delivery person receives the food from the cooking facility and delivers it to the location specified by the user. The input is the delivery instruction and the ordered food, and the output is the completion of the food delivery.

[0607] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0608] The system of the present invention not only eliminates food waste through the efficient collection, management, cooking, and delivery of surplus ingredients, but also enables the system to suggest optimal cooking menus based on the user's emotional state. The system includes a user terminal, a server, a generation AI, a database, food delivery, cooking facilities, and an emotion engine. The detailed process and specific embodiments are described below.

[0609] 1. Take a photo of your surplus food and send it to us

[0610] Users use a smartphone or other device to take a photo of surplus food items using a dedicated application. For example, they can take a photo of leftover tomatoes, peppers, onions, or other ingredients at home. After taking the photo, they can use the application's sending function to send the image to the server.

[0611] 2. Image analysis and database registration

[0612] The server uses a generation AI to analyze the received images. This generation AI uses image recognition technology to determine the type, size, quantity, and freshness of the ingredients. For example, it identifies a red, round object as a "tomato" and a green, elongated object as a "bell pepper." The analyzed ingredient information is then registered in a database.

[0613] 3. Instructions for collection and collection of ingredients

[0614] The server determines the optimal collection schedule and location based on the food ingredient information registered in the database. Collection instructions are sent to the food delivery person, informing them of the specific collection date, time, and location (e.g., User A's address). The delivery person visits the user's address at the specified date and time to collect the excess ingredients.

[0615] 4. Transporting ingredients and delivering them to cooking facilities

[0616] The delivery staff will then transport the collected ingredients to a partner cooking facility, where they will be inspected and cleaned.

[0617] 5. Create a food menu

[0618] The AI ​​on the server creates the optimal menu based on the ingredient information registered in the database. For example, it generates a recipe for "stir-fried vegetables with tomatoes, peppers, and onions." The generated menu and recipe are then sent to the cooking facility.

[0619] 6. Cooking and storage

[0620] The kitchen staff prepares the ingredients according to the recipes created by the generative AI, and once cooking is complete, the food is stored in the appropriate way, such as refrigerated or frozen.

[0621] 7. User Sentiment Analysis

[0622] The emotion engine installed in the user device analyzes the user's emotions. For example, it uses facial recognition technology and voice analysis to recognize the user's current emotions (happiness, sadness, stress, etc.).

[0623] 8. Menu suggestions based on emotion information

[0624] The server proposes optimal meal menus based on the user's emotional information analyzed by the emotion engine. For example, if the user is feeling stressed, it will propose meal menus using ingredients that have a relaxing effect. The meal menus are also displayed on the user's device, expanding the user's options.

[0625] 9. User Orders and Delivery

[0626] Users can order pre-cooked meals through a dedicated application. The application displays a list of available menu items, allowing users to select the desired dish and complete the order. The order information is sent to a server, which then sends delivery instructions to a delivery person. The delivery person collects the ordered food from the cooking facility and delivers it to the location specified by the user.

[0627] As a specific example, a user takes a photo of two surplus tomatoes, one bell pepper, and one onion at home and sends it to the server via the app. The server's generation AI analyzes the image and registers "two tomatoes, one bell pepper, and one onion" in a database. A delivery person receives collection instructions, goes to the user's home to collect the ingredients, and transports them to a cooking facility. The generation AI suggests "stir-fried vegetables," and the dish is prepared and stored at the cooking facility. If the emotion engine on the user's device analyzes that the user is feeling stressed, the server suggests a dish with a relaxing effect. Finally, the user orders the menu item via the app, and the delivery person delivers it to their home. This makes effective use of surplus ingredients, effectively eliminating food waste, and also meets the user's emotional needs.

[0628] The processing flow will be explained below.

[0629] Step 1:

[0630] The user launches the app and takes a photo of the surplus food items, such as tomatoes, peppers, or onions, using the camera function on their smartphone.

[0631] Step 2:

[0632] The device sends the captured image to the server. When the image data is sent, it is given a timestamp and user ID information.

[0633] Step 3:

[0634] The server receives the image and analyzes it using generative AI. Specifically, it uses an image recognition algorithm to determine the type, size, quantity, and freshness of the food. For example, it identifies a red, round object as a "tomato" and evaluates its freshness based on its size and color.

[0635] Step 4:

[0636] The server registers the analysis results in a database. Along with ingredient information (e.g., "2 tomatoes, 1 bell pepper, 1 onion"), detailed information such as freshness and the date and time the photo was taken is also recorded. User emotional information is also recorded here.

[0637] Step 5:

[0638] The device activates an emotion engine to analyze the user's emotions, for example, by using voice analysis and facial recognition technology to recognize the user's current emotions (happiness, sadness, stress, etc.).

[0639] Step 6:

[0640] The server determines the optimal collection schedule and location. Based on the information in the database, it sends collection instructions to the delivery person. These instructions include the date, time, and location of the food collection (e.g., User A's address).

[0641] Step 7:

[0642] The delivery person will collect the ingredients at the specified date, time and location. The delivery person will go to User A's address and pick up the surplus ingredients.

[0643] Step 8:

[0644] The delivery person transports the collected ingredients to the cooking facility. During the transport, the condition of the ingredients is easily checked and any problems are reported to the server.

[0645] Step 9:

[0646] The server communicates ingredient information to the cooking facility, including the type, quantity, and freshness of the ingredients.

[0647] Step 10:

[0648] The generative AI creates the optimal cooking menu. Based on the ingredient information in the database and the user's emotional information, it generates a recipe such as "Stir-fried vegetables with tomatoes, peppers, and onions" and sends it to the cooking facility.

[0649] Step 11:

[0650] The staff at the kitchen will prepare the food according to the recipes generated by the AI, and once the food is cooked, it will be stored appropriately, for example by refrigerating or freezing it.

[0651] Step 12:

[0652] Users order pre-cooked meals through a dedicated app, which displays available menu items, and users can select the desired dish to complete the order.

[0653] Step 13:

[0654] The server sends delivery instructions to the delivery person based on the order information. The delivery person is instructed to "Deliver the food from the cooking facility to User B's address."

[0655] Step 14:

[0656] The delivery person picks up the ordered food at the cooking facility and delivers it to the address of User B. After confirming that the food has been delivered safely, the delivery is reported to the server as completed.

[0657] As a specific example, a user takes a photo of two surplus tomatoes, one bell pepper, and one onion at home and sends it to a server via the app. The server's generation AI analyzes the image and registers "two tomatoes, one bell pepper, and one onion" in a database. The device's emotion engine analyzes the user's stress and registers this information in the database. A delivery person receives collection instructions, goes to the user's home, collects the ingredients, and transports them to a cooking facility. The generation AI suggests a "relaxing stir-fry vegetable dish" based on the user's emotions, and the dish is prepared and stored at the cooking facility. Finally, the user orders the menu item via the app, and the delivery person delivers it to their home. This makes effective use of surplus ingredients, effectively eliminating food waste, and also meets the user's emotional needs.

[0658] Example 2

[0659] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0660] In modern society, excessive food purchases and improper storage result in large amounts of surplus food, making food waste a serious problem. Furthermore, since it is difficult to provide meals that meet the diverse emotional needs of users, appropriate measures are required to improve user satisfaction. To solve these issues, an efficient and optimal system is required.

[0661] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0662] In this invention, the server includes a user terminal means for taking pictures of surplus ingredients, a generating AI means for analyzing the transmitted images and registering ingredient information in a database, and a means for proposing optimal recipes based on emotional information, thereby enabling effective use of surplus ingredients and proposing menus that meet the emotional needs of users.

[0663] "Surplus food" refers to food that is left unused in homes, restaurants, etc.

[0664] "User terminal" refers to a portable electronic device such as a smartphone or tablet used by a user.

[0665] "Generative AI" refers to artificial intelligence that uses machine learning algorithms to analyze data and perform specific tasks automatically.

[0666] A "database" refers to a system for efficiently storing, managing, and retrieving data.

[0667] "Food delivery" refers to a service that delivers pre-cooked food to the customer's home.

[0668] "Delivery person" refers to a person whose job is to deliver goods or food from a designated location to another.

[0669] "Cooking facility" means the place where food is prepared and stored.

[0670] "Emotion analysis" refers to the process of measuring and analyzing a user's emotional state using data such as facial expressions and voice.

[0671] "Cooking Menu" refers to a recipe or list of dishes prepared using specific ingredients.

[0672] "Proposal means" refers to techniques and methods that present optimal options based on the user's requests and circumstances.

[0673] This invention is a system that efficiently collects, analyzes, manages, cooks, and delivers surplus ingredients. It also has a function that suggests optimal cooking menus based on the user's emotional state. This system consists of a user terminal, a server, a generation AI, a database, food delivery, cooking facilities, and an emotion engine.

[0674] System configuration

[0675] 1. User Device

[0676] The user takes a photo of the surplus ingredients using a user device such as a smartphone or tablet, launches a dedicated application, takes a photo of the ingredients, and sends it to the server.

[0677] 2. Server

[0678] The server uses a generative AI model to analyze the received images. This generative AI model uses image recognition technology to determine the type, quantity, and freshness of ingredients. The analysis results are then registered in a database.

[0679] 3. Generative AI Models

[0680] The generative AI creates optimal recipes based on the collected information on surplus ingredients. For example, it can generate a recipe for "stir-fried vegetables with tomatoes, peppers, and onions."

[0681] 4. Database

[0682] The database stores and manages the analyzed ingredient information, recording information such as the type, quantity, and freshness of ingredients.

[0683] 5. Food delivery

[0684] Based on the food ingredient information in the database, the server instructs the food delivery service's delivery staff on the optimal collection schedule and location. The delivery staff follows these instructions, collects the surplus ingredients at the specified date, time, and location, and delivers them to a partner cooking facility.

[0685] 6. Cooking facilities

[0686] The food preparation facility receives the collected ingredients, checks and cleans them, and then prepares them according to the recipes provided by the AI. The cooked food is then stored in an appropriate manner, such as refrigerated or frozen.

[0687] 7. Emotion Engine

[0688] The emotion engine installed on the user's device uses facial recognition technology and voice analysis to analyze the user's current emotional state. The analysis results are sent to a server, which then suggests optimal cooking menus based on the user's emotions.

[0689] Examples of concrete examples and prompts

[0690] As a specific example, a user takes a photo of two leftover tomatoes, one green pepper, and one onion at home and sends it to a server via a dedicated app. The server's generation AI analyzes the image and registers "two tomatoes, one green pepper, and one onion" in a database. A delivery person receives collection instructions, goes to the user's home to collect the ingredients, and transports them to a cooking facility. The generation AI generates a recipe for "stir-fried vegetables," which is then cooked at the cooking facility. Furthermore, the emotion engine on the user's device analyzes that the user is in a stressful state, and the server suggests a dish menu with a relaxing effect. The user then orders the suggested menu using a dedicated app, and a delivery person delivers the food from the cooking facility to the user's home.

[0691] An example of a prompt is as follows:

[0692] "I have two tomatoes, one bell pepper, and one onion. Can you suggest a dish I can make using these ingredients?"

[0693] "Please suggest a meal plan when the user is feeling stressed."

[0694] This allows for effective use of surplus ingredients and menu suggestions that meet the emotional needs of users.

[0695] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0696] Step 1:

[0697] Input: Images of surplus food taken by a user using a user device such as a smartphone.

[0698] Specific operation: The user launches a dedicated application and takes a photo of surplus ingredients (e.g., two tomatoes, one green pepper, and one onion).

[0699] Output: The captured image is saved in the user's device.

[0700] Next process: The saved image is sent to the server.

[0701] Step 2:

[0702] Input: Ingredient images sent from the user device.

[0703] Specific operation: The user taps the send button in the application to send the captured image to the server.

[0704] Output: The image data arrives at the server.

[0705] Next process: The server analyzes the received image.

[0706] Step 3:

[0707] Input: Image data received by the server.

[0708] Specific operation: The server inputs the received image into the generative AI model and uses image recognition technology to analyze the type, quantity, and freshness of ingredients. For example, the generative AI recognizes a red, round shape as a "tomato."

[0709] Output: The analysis result is "2 tomatoes, 1 bell pepper, and 1 onion."

[0710] Next process: The analysis results are registered in a database.

[0711] Step 4:

[0712] Input: Parsed ingredient information.

[0713] Specific operation: The server registers the analysis results in a database.

[0714] Output: Stored in the database as "2 tomatoes, 1 bell pepper, 1 onion".

[0715] Next action: Collection instructions are generated from the database.

[0716] Step 5:

[0717] Input: Ingredient information stored in the database.

[0718] Specific operation: Based on the ingredient information in the database, the server instructs the food delivery person on the optimal collection schedule and location. For example, it may instruct the person to "collect two tomatoes, one green pepper, and one onion from User A's address."

[0719] Output: Collection instructions received by the courier.

[0720] Next step: The delivery person collects the ingredients.

[0721] Step 6:

[0722] Input: Collection instructions received by the courier.

[0723] Specific operation: The delivery person will go to the user's address at the specified date and time and collect the excess ingredients.

[0724] Output: Recovered ingredients.

[0725] Next process: The recovered ingredients are transported to a food preparation facility.

[0726] Step 7:

[0727] Input: Recovered ingredients.

[0728] Specific operation: The delivery person transports the collected ingredients to a partner cooking facility.

[0729] Output: Food arriving at the cooking facility.

[0730] Next step: the ingredients are cooked.

[0731] Step 8:

[0732] Input: Ingredients arriving at the cooking facility.

[0733] Specific actions: The kitchen staff inspects the collected ingredients, cleans them, and then cooks them according to the recipe provided by the generative AI. For example, they make a vegetable stir-fry using tomatoes, peppers, and onions.

[0734] Output: Cooked food.

[0735] Next process: The dish is saved.

[0736] Step 9:

[0737] Input: Cooked food.

[0738] Specific Behavior: Store cooked food in the correct manner, such as by refrigerating or freezing.

[0739] Output: Saved dishes.

[0740] Next process: Emotion analysis on the user device.

[0741] Step 10:

[0742] Input: The user's emotional state.

[0743] Specific operation: The emotion engine installed on the user's device analyzes the user's emotional state using facial recognition and voice analysis.

[0744] Output: Parsed emotion information.

[0745] Next process: Menu suggestions are made based on emotional information.

[0746] Step 11:

[0747] Input: Parsed emotion information.

[0748] Specific operation: The server proposes the most suitable food menu based on the emotional information analyzed by the emotion engine. For example, if the user is feeling stressed, it will propose dishes that have a relaxing effect.

[0749] Output: A menu of dishes suggested to the user.

[0750] Next: The user orders food.

[0751] Step 12:

[0752] Input: Suggested food menu.

[0753] Specific operation: The user uses a dedicated application to select the desired dish from the menu and complete the order.

[0754] Output: Order information.

[0755] Next step: The ordered food is delivered.

[0756] Step 13:

[0757] Input: User's order information.

[0758] Specific operation: The server receives the order information and sends delivery instructions to the delivery person. The delivery person picks up the ordered food from the cooking facility and delivers it to the location specified by the user.

[0759] Output: The food delivered to the user.

[0760] Next process: The user's meal is completed and the system process ends.

[0761] (Application example 2)

[0762] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0763] This invention aims to address the issue of food waste while improving users' life satisfaction by suggesting optimal cooking menus based on the user's emotional state. Conventional food delivery services have had difficulty making effective use of surplus ingredients or providing personalized services based on the user's emotional state. This has led to increased food waste and a poor user experience.

[0764] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a generation AI means that analyzes the transmitted image and registers the ingredient information in a database, an emotion engine means that analyzes the user's emotional state using the user terminal, and a server means that suggests a dish menu based on the user's emotional state. This makes it possible to effectively use surplus ingredients and suggest a dish menu that is optimal for the user's current emotional state.

[0765] "Surplus food" refers to food that is left over in homes or facilities and that may be discarded without being used.

[0766] A "user terminal" is a device that a user uses to take pictures of surplus ingredients and input order information, and includes smartphones, tablets, and the like.

[0767] The "generative AI means" is an artificial intelligence technology that analyzes the transmitted image, identifies ingredient information, and registers it in a database.

[0768] The "emotion engine means" is a technology for analyzing the user's emotional state, and includes facial recognition technology, voice analysis technology, and the like.

[0769] The "server means" is a central processing unit that issues collection instructions and suggests cooking menus based on ingredient information and the user's emotional state.

[0770] "Delivery personnel means" refers to personnel and vehicles that actually collect surplus ingredients and deliver cooked meals to users.

[0771] "Affiliated Food Facility" means a food facility or restaurant with which we have contracted to prepare and store recovered food.

[0772] The "means for suggesting a cooking menu" is a technology that allows the generative AI to create the optimal cooking menu based on the user's emotional state and ingredient information in the database.

[0773] A "database" is an electronic data storage device for centrally managing analyzed ingredient information, user order information, and the like.

[0774] Overall system overview

[0775] This invention is a system that efficiently collects and manages surplus ingredients, and proposes and delivers meal plans tailored to the user's emotional state. The following describes in detail how this system is implemented.

[0776] Hardware and Software Use

[0777] This system mainly uses the following hardware and software:

[0778] Smartphone: Used to take photos of surplus ingredients and analyze user emotions

[0779] Server: Used for image analysis, food ingredient information registration using generative AI, user emotional state analysis, and menu suggestions

[0780] Generative AI model: Analyzes ingredient information and generates optimal cooking menus

[0781] Emotion engine: Analyzes the user's emotional state through facial recognition and voice analysis

[0782] Database: Stores information about ingredients and user orders

[0783] Explanation of program processing

[0784] 1. Photograph and send your surplus ingredients:

[0785] Users use a dedicated smartphone app to take a photo of leftover ingredients at home, and the app sends the image to a server.

[0786] 2. Image analysis and database registration:

[0787] The server uses a generative AI model to analyze the received images, identifying the type, quantity, and freshness of ingredients from the images and registering this information in a database.

[0788] 3. Instructions for collection and collection of ingredients:

[0789] The server generates an optimal collection schedule based on the food ingredient information registered in the database and sends instructions to the delivery staff, who then collect the ingredients at the specified date and time and transport them to the partner cooking facility.

[0790] 4. Create a food menu:

[0791] The generative AI model on the server creates the optimal recipe based on the ingredient information registered in the database. For example, it generates a recipe for "stir-fried vegetables with tomatoes, peppers, and onions."

[0792] 5. Cooking and storage:

[0793] Staff at partner kitchens will prepare the ingredients according to the recipes created by the generative AI, and the cooked food will be stored in an appropriate manner, such as refrigerated or frozen.

[0794] 6. User sentiment analysis:

[0795] The emotion engine on the user device analyzes the user's emotional state using a camera and microphone, and the analyzed emotion information is sent to the server.

[0796] 7. Menu suggestions based on emotion information:

[0797] The server uses a generative AI model to suggest optimal meal plans based on the analyzed emotional information. For example, if the user is feeling stressed, the server suggests meal plans with a relaxing effect.

[0798] 8. User Orders and Delivery:

[0799] The user places an order by selecting from the menu of suggested dishes through a dedicated app. The server receives the order information and sends delivery instructions to the delivery person. The delivery person then picks up the food from the cooking facility and delivers it to the user.

[0800] Specific examples

[0801] Specifically, User A takes a photo of two tomatoes, one green pepper, and one onion that they have left over at home and sends it to the server via the app. The server's generation AI analyzes this and registers "two tomatoes, one green pepper, and one onion" in a database. A delivery person then receives collection instructions, goes to User A's home to collect the ingredients, and transports them to a cooking facility. The generation AI suggests "stir-fried vegetables," and the dish is prepared at the cooking facility. When the emotion engine on the user's device analyzes that User A is feeling stressed, the server suggests "stir-fried vegetables with sesame flavor" as a dish that has a relaxing effect. Finally, User A orders this menu through the app, and a delivery person delivers it to their home. This makes it possible to make effective use of surplus ingredients, reduce food waste, and provide a service that is tailored to the user's emotions.

[0802] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0803] Step 1:

[0804] When a user realizes that they have surplus ingredients at home, they take a photo of the ingredients using a dedicated smartphone app. The input is the photo of the ingredients, and the output is image data stored on the smartphone. The smartphone stores this image data within the application.

[0805] Step 2:

[0806] A smartphone application sends a captured image to a server. The input is image data, and the output is the sent image data. Specifically, the application uploads the image file to the server using an HTTP request.

[0807] Step 3:

[0808] The server uses a generative AI model to analyze the transmitted image. The input is image data, and the output is analyzed ingredient information (type, quantity, freshness, etc.). The generative AI model analyzes the data using image recognition technology to identify ingredient information.

[0809] Step 4:

[0810] The server registers the analyzed ingredient information in a database. The input is the analyzed ingredient information, and the output is a record to be saved in the database. The server inserts the ingredient information into the database using an SQL query.

[0811] Step 5:

[0812] The server generates an optimal collection schedule based on the food ingredient information registered in the database and sends collection instructions to the delivery staff. The input is the food ingredient information in the database, and the output is the collection instructions sent to the delivery staff. The server uses a scheduling algorithm to determine the optimal collection date, time, and location.

[0813] Step 6:

[0814] The delivery person visits the user's address at the specified date and time to collect the excess ingredients. The input is the collection instruction from the server, and the output is the collected ingredients. The delivery person safely transports the ingredients using a container specially designed for food.

[0815] Step 7:

[0816] The delivery staff transports the collected ingredients to the partner cooking facility. The input is the collected ingredients, and the output is the ingredients delivered to the cooking facility. The delivery staff uses appropriate transportation methods to ensure the ingredients are kept fresh.

[0817] Step 8:

[0818] The generative AI model on the server creates the optimal cooking menu based on the ingredient information registered in the database. The input is the ingredient information in the database, and the output is the generated cooking menu. The generative AI model generates the optimal recipe by taking into account the combination of ingredients and cooking methods.

[0819] Step 9:

[0820] Partnered kitchens prepare ingredients according to recipes created by the generative AI. The input is the generated menu and ingredients, and the output is the cooked food. Kitchen staff follow the set procedures to create high-quality dishes.

[0821] Step 10:

[0822] The emotion engine on the user device analyzes the user's emotional state. The input is the user's facial image and voice data, and the output is analyzed emotional information. The emotion engine recognizes the user's emotions using facial recognition technology and voice analysis technology.

[0823] Step 11:

[0824] The server uses a generative AI model to propose optimal cooking menus based on the analyzed emotional information. The input is emotional information and ingredient information, and the output is an optimal cooking menu based on the emotions. The server proposes menus that have a relaxing effect or that are invigorating depending on the emotions.

[0825] Step 12:

[0826] Users place their orders by selecting from the proposed menu through a dedicated app. The input is the menu information, and the output is the user's order data. The app provides an interface that allows users to easily select menu items and complete their orders.

[0827] Step 13:

[0828] The server receives the user's order information and sends delivery instructions to the delivery person. The input is the user's order information and the output is delivery instructions. The server generates the optimal delivery route and schedule and sends the delivery instructions to the delivery person.

[0829] Step 14:

[0830] The delivery person receives the food from the cooking facility and delivers it to the location specified by the user. The input is the cooked food and delivery instructions, and the output is the food delivered to the user. The delivery person is required to deliver the ordered food safely and quickly.

[0831] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0832] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0833] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0834] [Third embodiment]

[0835] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0836] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0837] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0838] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0839] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0840] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0841] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0842] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0843] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0844] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0845] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0846] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0847] The system of the present invention provides a comprehensive solution to eliminate food waste through the efficient collection, management, cooking, and delivery of surplus food ingredients. The detailed process and specific embodiments are described below.

[0848] 1. Take a photo of your surplus food and send it to us

[0849] Users use a smartphone or other device to take a photo of surplus food items using a dedicated application. For example, they can take a photo of leftover tomatoes, peppers, onions, or other ingredients at home. After taking the photo, they can use the application's sending function to send the image to the server.

[0850] 2. Image analysis and database registration

[0851] The server uses a generation AI to analyze the received images. This generation AI uses image recognition technology to determine the type, size, quantity, and freshness of the ingredients. For example, it identifies a red, round object as a "tomato" and a green, elongated object as a "bell pepper." The analyzed ingredient information is then registered in a database.

[0852] 3. Instructions for collection and collection of ingredients

[0853] The server determines the optimal collection schedule and location based on the food ingredient information registered in the database. Collection instructions are sent to the food delivery person, informing them of the specific collection date, time, and location (e.g., User A's address). The delivery person visits the user's address at the specified date and time to collect the excess ingredients.

[0854] 4. Transporting ingredients and delivering them to cooking facilities

[0855] The delivery staff will then transport the collected ingredients to a partner cooking facility, where they will be inspected and cleaned.

[0856] 5. Create a food menu

[0857] The AI ​​on the server creates the optimal menu based on the ingredient information registered in the database. For example, it generates a recipe for "stir-fried vegetables with tomatoes, peppers, and onions." The generated menu and recipe are then sent to the cooking facility.

[0858] 6. Cooking and storage

[0859] The kitchen staff prepares the ingredients according to the recipes created by the generative AI, and once cooking is complete, the food is stored in the appropriate way, such as refrigerated or frozen.

[0860] 7. User Orders and Delivery

[0861] Users can order pre-cooked meals through a dedicated application. The application displays a list of available menu items, allowing users to select the desired dish and complete the order. The order information is sent to a server, which then sends delivery instructions to a delivery person. The delivery person collects the ordered food from the cooking facility and delivers it to the location specified by the user.

[0862] As a specific example, a user takes a photo of two tomatoes, one green pepper, and one onion that they have left over at home and sends it to the server via the app. The server's generation AI analyzes the photo and registers "two tomatoes, one green pepper, and one onion" in the database. A delivery person receives collection instructions, goes to the user's home to collect the ingredients, and transports them to a cooking facility. The generation AI suggests "stir-fried vegetables," and the dish is prepared and stored at the cooking facility. Finally, the user orders the menu item via the app, and the delivery person delivers it to their home. This makes effective use of surplus ingredients and effectively eliminates food waste.

[0863] The processing flow will be explained below.

[0864] Step 1:

[0865] The user launches the app and takes a photo of the surplus food items, such as tomatoes, peppers, or onions, using the camera function on their smartphone.

[0866] Step 2:

[0867] The device sends the captured image to the server. When the image data is sent, it is given a timestamp and user ID information.

[0868] Step 3:

[0869] The server receives the image and analyzes it using generative AI. Specifically, it uses an image recognition algorithm to determine the type, size, quantity, and freshness of the food. For example, it identifies a red, round object as a "tomato" and evaluates its freshness based on its size and color.

[0870] Step 4:

[0871] The server registers the analysis results in a database, recording detailed information such as the freshness of the ingredients (e.g., "2 tomatoes, 1 bell pepper, 1 onion") and the date and time the photo was taken.

[0872] Step 5:

[0873] The server determines the optimal collection schedule and location. Based on the information in the database, it sends collection instructions to the delivery person. These instructions include the date, time, and location of the food collection (e.g., User A's address).

[0874] Step 6:

[0875] The delivery person will collect the ingredients at the specified date, time and location. The delivery person will go to User A's address and pick up the surplus ingredients.

[0876] Step 7:

[0877] The delivery person transports the collected ingredients to the cooking facility. During the transport, the condition of the ingredients is easily checked and any problems are reported to the server.

[0878] Step 8:

[0879] The server communicates ingredient information to the cooking facility, including the type, quantity, and freshness of the ingredients.

[0880] Step 9:

[0881] The generative AI creates the optimal menu, generating recipes such as "stir-fried vegetables with tomatoes, peppers, and onions" based on the ingredients in the database, and sending them to the cooking facility.

[0882] Step 10:

[0883] The staff at the kitchen will prepare the food according to the recipes generated by the AI, and once the food is cooked, it will be stored appropriately, for example by refrigerating or freezing it.

[0884] Step 11:

[0885] Users order pre-cooked meals through a dedicated app, which displays available menu items, and users can select the desired dish to complete the order.

[0886] Step 12:

[0887] The server sends delivery instructions to the delivery person based on the order information. The delivery person is instructed to "Deliver the food from the cooking facility to User B's address."

[0888] Step 13:

[0889] The delivery person picks up the ordered food at the cooking facility and delivers it to the address of User B. After confirming that the food has been delivered safely, the delivery is reported to the server as completed.

[0890] Example 1

[0891] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0892] Currently, the effective use of surplus ingredients and the problem of food waste are social issues, but efficient systems for collection, management, cooking, and delivery are not fully established. There is also a lack of advanced analytical technology to determine the freshness of ingredients and propose appropriate cooking menus. For this reason, a system that can effectively utilize surplus ingredients is needed.

[0893] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0894] In this invention, the server includes a user terminal means for taking pictures of surplus ingredients, a means for sending the taken images to the server, and a generating AI means on the server for analyzing the sent images and registering ingredient information in a database, thereby enabling efficient collection, analysis, and management of surplus ingredients.

[0895] The server also includes a means for sending a collection instruction to a delivery person based on the ingredient information, and a delivery person means for collecting the ingredients according to the collection instruction and transporting them to the cooking facility, thereby enabling the ingredients to be collected and delivered to the cooking facility quickly.

[0896] Furthermore, the server includes a generation AI means for creating an optimal menu based on the ingredient information in the database, a means for cooking and storing ingredients at a cooking facility, a delivery staff means for delivering cooked food based on the user's order, a means for reporting the status of the food en route to the cooking facility to the server in real time, and a means for updating the server with information on completion of storage, thereby enabling the effective use of ingredients, appropriate cooking and storage, and prompt delivery to the user.

[0897] "Surplus ingredients" refers to ingredients or food that are left over and not consumed.

[0898] "User terminal means" refers to a device used by a user, such as a computer or smartphone, that has the function of photographing ingredients and transmitting information about them.

[0899] "Server" refers to a computer system that processes, stores, and transmits data over a network.

[0900] "Generative AI means" refers to technologies that use artificial intelligence (AI) to analyze data and generate specific functions or results.

[0901] "Delivery means" refers to the person or system that collects ingredients or food from a designated location and transports them to a designated location.

[0902] "Ingredient information" refers to data including details such as the type, quantity, and freshness of ingredients.

[0903] A "database" is a data collection point and a system for organizing, storing, and searching various information.

[0904] "Cooking facility" refers to a facility that receives, cooks, and preserves food ingredients.

[0905] "Food Menu" means a list of the types of food available and their details.

[0906] "Freshness" refers to the degree of freshness of food or ingredients, and is usually a standard for evaluating quality, expiration dates, etc.

[0907] "Preservation measures" refers to measures such as refrigeration, freezing, and packaging that are used to store cooked food and ingredients in an appropriate state.

[0908] The system of the present invention provides a comprehensive solution to eliminate food waste through the efficient collection, management, cooking, and delivery of surplus ingredients. Specific embodiments of the system of the present invention are described below.

[0909] Using a smartphone or other user device, a user can take a photo of surplus food items using a dedicated application. For example, they can take a photo of leftover tomatoes, peppers, onions, or other ingredients at home, and press the send button on the application to send the image to the server.

[0910] The server then launches a generative AI model to analyze the received images. This generative AI model uses image recognition technologies such as TensorFlow and PyTorch to determine the type, size, quantity, and freshness of the ingredients and registers them in a database. For example, it might identify a red, round object as a "tomato" and a green, elongated object as a "bell pepper."

[0911] Based on the food ingredient information registered in the database, the server determines the optimal collection schedule and location. Collection instructions are sent to the delivery person's device, and the delivery person visits the user's address at the specified date and time to collect the excess food. The GPS-equipped device provides the delivery person with the optimal route, supporting efficient collection.

[0912] The delivery person transports the collected ingredients to the cooking facility. During this process, the status of the delivery is reported to the server in real time. Upon arrival at the cooking facility, information confirming the arrival of the ingredients is sent to the server. The cooking facility staff checks the ingredients and cleans up if necessary.

[0913] The generative AI model on the server generates the optimal dish menu based on the ingredient information registered in the database. This process uses a natural language generation model (e.g., GPT-4). For example, a recipe for "stir-fried vegetables with tomatoes, peppers, and onions" is generated and sent to the cooking facility.

[0914] The staff at the kitchen prepares the ingredients according to the recipe created by the generative AI. Once the food is cooked, it is stored in an appropriate manner, such as refrigerated or frozen. This information about the completion of storage is updated to the server.

[0915] Users can order pre-cooked meals through a dedicated application. The application displays a list of available menu items, and users can select the desired dish to complete the order. The order information is sent to a server, which then sends delivery instructions to a delivery person. The delivery person collects the ordered food from the cooking facility and delivers it to the address specified by the user.

[0916] Examples:

[0917] The user takes a photo of two surplus tomatoes, one bell pepper, and one onion at home and sends it to the server via the app. The server's generation AI analyzes the photo and registers it in the database as "two tomatoes, one bell pepper, and one onion." A delivery person receives collection instructions, goes to the user's home to collect the ingredients, and transports them to a cooking facility. The generation AI suggests a recipe for "stir-fried vegetables," and the dish is prepared at the cooking facility based on this recipe and stored. Finally, the user orders the menu item via the app, and the delivery person delivers it to their home. This makes effective use of surplus ingredients and effectively eliminates food waste.

[0918] Example prompt for a generative AI model:

[0919] "Please suggest the best recipe based on the ingredient information received from the user. For example, if two tomatoes, one bell pepper, and one onion are provided, please consider what kind of dish can be made using them and provide a detailed recipe."

[0920] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0921] Step 1:

[0922] The user uses a dedicated smartphone application to take a photo of surplus ingredients. For example, they can take a photo of leftover tomatoes, peppers, onions, and other ingredients at home. The input is an image of each ingredient, and the output is an image file.

[0923] Step 2:

[0924] The user presses the send button in the dedicated application to send the captured image to the server. The input is the image file obtained in step 1, and the output is a notification that the image file has been uploaded to the server.

[0925] Step 3:

[0926] The server launches a generative AI model to analyze the received image. The input is the transmitted image file, and the generative AI model performs image recognition using TensorFlow or PyTorch. The output is the recognized ingredient information (e.g., 2 tomatoes, 1 bell pepper, 1 onion). The server registers this information in a database.

[0927] Step 4:

[0928] The server determines the optimal collection schedule and location based on the food ingredient information registered in the database. The input is the food ingredient information registered in the database, and the output is collection instructions to the delivery person. The collection instructions include the specific collection date, time, and location.

[0929] Step 5:

[0930] The delivery person visits the user's address at the specified date and time to collect the excess ingredients. The input is the collection instruction from the server, and the output is the collected ingredients. The delivery person's device is equipped with a GPS, which provides the optimal route.

[0931] Step 6:

[0932] The delivery person transports the collected ingredients to the cooking facility and reports the status of the delivery to the server in real time. The input is the collected ingredients, and the output is a notification of arrival at the cooking facility.

[0933] Step 7:

[0934] The kitchen staff checks the food as it arrives and cleans it up as necessary. The input is the delivered food, and the output is the cleaned food.

[0935] Step 8:

[0936] A generative AI model on the server generates optimal cooking menus based on ingredient information registered in a database. The input is ingredient information from the database, and the generative AI model uses natural language generation technologies such as GPT-4. The output is the generated cooking menu and recipe.

[0937] Step 9:

[0938] The kitchen staff prepares the ingredients according to the recipes created by the generative AI model. The input is the generated food menu and recipe, and the output is the cooked food.

[0939] Step 10:

[0940] Once the food is cooked, it is stored in an appropriate way, such as refrigerated or frozen. The input is the cooked food, and the output is the stored food. This storage completion information is updated to the server.

[0941] Step 11:

[0942] A user orders cooked food using a dedicated application. The input is the user's order information, and the output is a notification that the order information has been sent to the server. The application displays a list of available menu items.

[0943] Step 12:

[0944] The server sends the order information to the delivery person as delivery instructions. The input is the user's order information, and the output is delivery instructions to the delivery person.

[0945] Step 13:

[0946] The delivery person receives the ordered food from the food preparation facility and delivers it to the address specified by the user. The input is the delivery instruction and the cooked food, and the output is a notification to the user that the food has been delivered.

[0947] (Application example 1)

[0948] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0949] A comprehensive system is needed to effectively eliminate food waste by efficiently collecting, managing, cooking, and delivering surplus ingredients. It is also required to generate optimal menus based on ingredient information, allowing users to easily order. However, current systems lack sufficient information on ingredient freshness and generate optimal collection schedules, making efficient operation difficult.

[0950] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0951] In this invention, the server includes a means for taking a photo of surplus ingredients using a user terminal and sending the image, a generating AI model means for analyzing the sent image and registering ingredient information in a database, and a means for generating food delivery instructions based on the ingredient information and sending them to a delivery person, thereby enabling the efficient collection, management, cooking, and delivery of surplus ingredients.

[0952] A "user terminal" is a device used to take pictures of surplus ingredients and send the images. This includes smartphones.

[0953] "Image analysis" is the process of analyzing the transmitted image and extracting ingredient information.

[0954] A "generative AI model" is a system that uses machine learning algorithms to create cooking menus based on image analysis and ingredient information.

[0955] A "database" is an information system that registers and manages analyzed ingredient information.

[0956] "Food delivery" is a service that delivers food and ingredients to a location specified by the user.

[0957] "Delivery Instructions" are instructions that indicate the specific schedule and location for collecting and delivering ingredients and food.

[0958] "Cooking facilities" are facilities for cooking and storing collected food ingredients.

[0959] A "cooking menu" is a recipe or list of dishes created based on ingredient information in the database.

[0960] "User order" is an operation performed by a user to order a cooked dish.

[0961] A "smartphone application" is an app that allows users to check and order information about ingredients and menu items.

[0962] This invention is a comprehensive system for eliminating food waste through the efficient collection, management, cooking, and delivery of surplus food ingredients. The system is composed of the following elements:

[0963] 1. User Device:

[0964] Users take photos of leftover ingredients using a user device such as a smartphone. The images are then sent to a server via a dedicated application. For example, users can take photos of leftover ingredients such as tomatoes, peppers, and onions at home.

[0965] 2. Image analysis and database registration:

[0966] The server uses a generative AI model to analyze the received images. This generative AI model uses image recognition technology to determine the type, size, quantity, and freshness of ingredients. For example, it might identify a red, round object as a "tomato" and a green, elongated object as a "bell pepper." The analyzed ingredient information is then registered in a database.

[0967] 3. Ingredient collection and scheduling:

[0968] The server determines the optimal collection schedule and location based on the food ingredient information registered in the database. Collection instructions are sent to the food delivery person, informing them of the specific collection date, time, and location (e.g., the user's address). The delivery person visits the user's address at the specified date and time to collect the excess ingredients.

[0969] 4. Transportation and delivery of ingredients to cooking facilities:

[0970] The delivery staff will then transport the collected ingredients to a partner cooking facility, where they will be inspected and cleaned.

[0971] 5. Create a food menu:

[0972] The generative AI model on the server creates the optimal cooking menu based on the ingredient information registered in the database. For example, it generates a recipe for "stir-fried vegetables with tomatoes, peppers, and onions." The generated menu and recipe are sent to the cooking facility.

[0973] 6. Preparation and storage:

[0974] The kitchen staff prepares the ingredients according to the recipes created by the generative AI, and once cooking is complete, the food is stored in the appropriate way, such as refrigerated or frozen.

[0975] 7. User Orders and Delivery:

[0976] Users can order pre-cooked meals through a smartphone application. The application displays a list of available menu items, and users can select the desired dish to complete the order. The order information is sent to a server, which then sends delivery instructions to a delivery person. The delivery person collects the ordered food from the cooking facility and delivers it to the location specified by the user.

[0977] Hardware and software used:

[0978] Smartphone: Used by users to take pictures of ingredients and operate applications.

[0979] Server: Performs central processing such as image analysis, database management, and delivery instruction generation.

[0980] Generative AI models: Includes image recognition technologies such as Google Cloud Vision API.

[0981] Cooking facilities: Facilities for cooking and storing recovered food.

[0982] Examples:

[0983] For example, a user takes a photo of two tomatoes, one green pepper, and one onion that they have left over at home and sends it to the server via the app. The generative AI model on the server analyzes the image and registers "two tomatoes, one green pepper, and one onion" in the database. A delivery person receives collection instructions, goes to the user's home to collect the ingredients, and transports them to a cooking facility. The generative AI model suggests "stir-fried vegetables," and the dish is prepared and stored at the cooking facility. Finally, the user orders the menu item via the app, and the delivery person delivers it to their home. This makes effective use of surplus ingredients and effectively eliminates food waste.

[0984] Example prompt sentence:

[0985] Image analysis prompt:

[0986] "Take a picture of leftover food in your home and analyze this image. Identify specific ingredients like tomatoes, peppers, and onions."

[0987] Recipe generation prompt:

[0988] "Generate the best recipe using the following ingredients: 2 tomatoes, 1 bell pepper, and 1 onion."

[0989] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0990] Step 1:

[0991] The user takes a photo of the surplus ingredients using a device such as a smartphone. For example, the user takes a photo of a tomato, pepper, or onion. This provides the image of the ingredients as input data.

[0992] Step 2:

[0993] The captured image is sent to the server. The user uses a dedicated application on their device to send the captured image of the ingredients to the server. The input data is the image of the ingredients, and processing begins when the server receives it.

[0994] Step 3:

[0995] The server analyzes the image. The server uses a generative AI model (such as Google Cloud Vision API) to analyze the received image. Information such as the type, size, quantity, and freshness of the ingredients is extracted from the image, and ingredient information is obtained from the input image data.

[0996] Step 4:

[0997] The ingredient information is registered in a database. The server registers the analyzed ingredient information in the database. Information such as the type and quantity of ingredients is saved in the database. The input is the analyzed ingredient information, and the output is the registration result in the database.

[0998] Step 5:

[0999] Generates an optimal collection schedule. The server determines the collection date, time, and location based on the food ingredient information registered in the database. The input data is the food ingredient information in the database, and the server generates an optimal collection schedule and notifies the delivery person as output.

[1000] Step 6:

[1001] The delivery person collects the ingredients. The delivery person follows instructions from the server and goes to the specified date, time and location to collect the ingredients. This allows the collected ingredients to be obtained from the user. The input is the server's collection instructions, and the output is the collected ingredients.

[1002] Step 7:

[1003] The collected ingredients are transported to the cooking facility. The delivery person transports the collected ingredients to the partner cooking facility. The input is the collected ingredients, and the output is the completion of transportation to the cooking facility.

[1004] Step 8:

[1005] The server generates a cooking menu. The generative AI model on the server creates the optimal cooking menu based on the ingredient information registered in the database. The input data is the ingredient information in the database, and the output is the generated cooking menu.

[1006] Step 9:

[1007] The food is cooked and stored in the kitchen. The staff at the kitchen follow the recipe generated by the server to prepare the food. After cooking, the food is stored by refrigeration or freezing. The input is the generated recipe, and the output is the cooked food.

[1008] Step 10:

[1009] The server accepts orders from users and issues delivery instructions. Users order pre-cooked food through a smartphone application, and the server issues instructions to the delivery person. The input is the user's order information, and the output is delivery instructions.

[1010] Step 11:

[1011] The delivery person delivers the ordered food to the user. The delivery person receives the food from the cooking facility and delivers it to the location specified by the user. The input is the delivery instruction and the ordered food, and the output is the completion of the food delivery.

[1012] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1013] The system of the present invention not only eliminates food waste through the efficient collection, management, cooking, and delivery of surplus ingredients, but also enables the system to suggest optimal cooking menus based on the user's emotional state. The system includes a user terminal, a server, a generation AI, a database, food delivery, cooking facilities, and an emotion engine. The detailed process and specific embodiments are described below.

[1014] 1. Take a photo of your surplus food and send it to us

[1015] Users use a smartphone or other device to take a photo of surplus food items using a dedicated application. For example, they can take a photo of leftover tomatoes, peppers, onions, or other ingredients at home. After taking the photo, they can use the application's sending function to send the image to the server.

[1016] 2. Image analysis and database registration

[1017] The server uses a generation AI to analyze the received images. This generation AI uses image recognition technology to determine the type, size, quantity, and freshness of the ingredients. For example, it identifies a red, round object as a "tomato" and a green, elongated object as a "bell pepper." The analyzed ingredient information is then registered in a database.

[1018] 3. Instructions for collection and collection of ingredients

[1019] The server determines the optimal collection schedule and location based on the food ingredient information registered in the database. Collection instructions are sent to the food delivery person, informing them of the specific collection date, time, and location (e.g., User A's address). The delivery person visits the user's address at the specified date and time to collect the excess ingredients.

[1020] 4. Transporting ingredients and delivering them to cooking facilities

[1021] The delivery staff will then transport the collected ingredients to a partner cooking facility, where they will be inspected and cleaned.

[1022] 5. Create a food menu

[1023] The AI ​​on the server creates the optimal menu based on the ingredient information registered in the database. For example, it generates a recipe for "stir-fried vegetables with tomatoes, peppers, and onions." The generated menu and recipe are then sent to the cooking facility.

[1024] 6. Cooking and storage

[1025] The kitchen staff prepares the ingredients according to the recipes created by the generative AI, and once cooking is complete, the food is stored in the appropriate way, such as refrigerated or frozen.

[1026] 7. User Sentiment Analysis

[1027] The emotion engine installed in the user device analyzes the user's emotions. For example, it uses facial recognition technology and voice analysis to recognize the user's current emotions (happiness, sadness, stress, etc.).

[1028] 8. Menu suggestions based on emotion information

[1029] The server proposes optimal meal menus based on the user's emotional information analyzed by the emotion engine. For example, if the user is feeling stressed, it will propose meal menus using ingredients that have a relaxing effect. The meal menus are also displayed on the user's device, expanding the user's options.

[1030] 9. User Orders and Delivery

[1031] Users can order pre-cooked meals through a dedicated application. The application displays a list of available menu items, allowing users to select the desired dish and complete the order. The order information is sent to a server, which then sends delivery instructions to a delivery person. The delivery person collects the ordered food from the cooking facility and delivers it to the location specified by the user.

[1032] As a specific example, a user takes a photo of two surplus tomatoes, one bell pepper, and one onion at home and sends it to the server via the app. The server's generation AI analyzes the image and registers "two tomatoes, one bell pepper, and one onion" in a database. A delivery person receives collection instructions, goes to the user's home to collect the ingredients, and transports them to a cooking facility. The generation AI suggests "stir-fried vegetables," and the dish is prepared and stored at the cooking facility. If the emotion engine on the user's device analyzes that the user is feeling stressed, the server suggests a dish with a relaxing effect. Finally, the user orders the menu item via the app, and the delivery person delivers it to their home. This makes effective use of surplus ingredients, effectively eliminating food waste, and also meets the user's emotional needs.

[1033] The processing flow will be explained below.

[1034] Step 1:

[1035] The user launches the app and takes a photo of the surplus food items, such as tomatoes, peppers, or onions, using the camera function on their smartphone.

[1036] Step 2:

[1037] The device sends the captured image to the server. When the image data is sent, it is given a timestamp and user ID information.

[1038] Step 3:

[1039] The server receives the image and analyzes it using generative AI. Specifically, it uses an image recognition algorithm to determine the type, size, quantity, and freshness of the food. For example, it identifies a red, round object as a "tomato" and evaluates its freshness based on its size and color.

[1040] Step 4:

[1041] The server registers the analysis results in a database. Along with ingredient information (e.g., "2 tomatoes, 1 bell pepper, 1 onion"), detailed information such as freshness and the date and time the photo was taken is also recorded. User emotional information is also recorded here.

[1042] Step 5:

[1043] The device activates an emotion engine to analyze the user's emotions, for example, by using voice analysis and facial recognition technology to recognize the user's current emotions (happiness, sadness, stress, etc.).

[1044] Step 6:

[1045] The server determines the optimal collection schedule and location. Based on the information in the database, it sends collection instructions to the delivery person. These instructions include the date, time, and location of the food collection (e.g., User A's address).

[1046] Step 7:

[1047] The delivery person will collect the ingredients at the specified date, time and location. The delivery person will go to User A's address and pick up the surplus ingredients.

[1048] Step 8:

[1049] The delivery person transports the collected ingredients to the cooking facility. During the transport, the condition of the ingredients is easily checked and any problems are reported to the server.

[1050] Step 9:

[1051] The server communicates ingredient information to the cooking facility, including the type, quantity, and freshness of the ingredients.

[1052] Step 10:

[1053] The generative AI creates the optimal cooking menu. Based on the ingredient information in the database and the user's emotional information, it generates a recipe such as "Stir-fried vegetables with tomatoes, peppers, and onions" and sends it to the cooking facility.

[1054] Step 11:

[1055] The staff at the kitchen will prepare the food according to the recipes generated by the AI, and once the food is cooked, it will be stored appropriately, for example by refrigerating or freezing it.

[1056] Step 12:

[1057] Users order pre-cooked meals through a dedicated app, which displays available menu items, and users can select the desired dish to complete the order.

[1058] Step 13:

[1059] The server sends delivery instructions to the delivery person based on the order information. The delivery person is instructed to "Deliver the food from the cooking facility to User B's address."

[1060] Step 14:

[1061] The delivery person picks up the ordered food at the cooking facility and delivers it to the address of User B. After confirming that the food has been delivered safely, the delivery is reported to the server as completed.

[1062] As a specific example, a user takes a photo of two surplus tomatoes, one bell pepper, and one onion at home and sends it to a server via the app. The server's generation AI analyzes the image and registers "two tomatoes, one bell pepper, and one onion" in a database. The device's emotion engine analyzes the user's stress and registers this information in the database. A delivery person receives collection instructions, goes to the user's home, collects the ingredients, and transports them to a cooking facility. The generation AI suggests a "relaxing stir-fry vegetable dish" based on the user's emotions, and the dish is prepared and stored at the cooking facility. Finally, the user orders the menu item via the app, and the delivery person delivers it to their home. This makes effective use of surplus ingredients, effectively eliminating food waste, and also meets the user's emotional needs.

[1063] Example 2

[1064] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1065] In modern society, excessive food purchases and improper storage result in large amounts of surplus food, making food waste a serious problem. Furthermore, since it is difficult to provide meals that meet the diverse emotional needs of users, appropriate measures are required to improve user satisfaction. To solve these issues, an efficient and optimal system is required.

[1066] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1067] In this invention, the server includes a user terminal means for taking pictures of surplus ingredients, a generating AI means for analyzing the transmitted images and registering ingredient information in a database, and a means for proposing optimal recipes based on emotional information, thereby enabling effective use of surplus ingredients and proposing menus that meet the emotional needs of users.

[1068] "Surplus food" refers to food that is left unused in homes, restaurants, etc.

[1069] "User terminal" refers to a portable electronic device such as a smartphone or tablet used by a user.

[1070] "Generative AI" refers to artificial intelligence that uses machine learning algorithms to analyze data and perform specific tasks automatically.

[1071] A "database" refers to a system for efficiently storing, managing, and retrieving data.

[1072] "Food delivery" refers to a service that delivers pre-cooked food to the customer's home.

[1073] "Delivery person" refers to a person whose job is to deliver goods or food from a designated location to another.

[1074] "Cooking facility" means the place where food is prepared and stored.

[1075] "Emotion analysis" refers to the process of measuring and analyzing a user's emotional state using data such as facial expressions and voice.

[1076] "Cooking Menu" refers to a recipe or list of dishes prepared using specific ingredients.

[1077] "Proposal means" refers to techniques and methods that present optimal options based on the user's requests and circumstances.

[1078] This invention is a system that efficiently collects, analyzes, manages, cooks, and delivers surplus ingredients. It also has a function that suggests optimal cooking menus based on the user's emotional state. This system consists of a user terminal, a server, a generation AI, a database, food delivery, cooking facilities, and an emotion engine.

[1079] System configuration

[1080] 1. User Device

[1081] The user takes a photo of the surplus ingredients using a user device such as a smartphone or tablet, launches a dedicated application, takes a photo of the ingredients, and sends it to the server.

[1082] 2. Server

[1083] The server uses a generative AI model to analyze the received images. This generative AI model uses image recognition technology to determine the type, quantity, and freshness of ingredients. The analysis results are then registered in a database.

[1084] 3. Generative AI Models

[1085] The generative AI creates optimal recipes based on the collected information on surplus ingredients. For example, it can generate a recipe for "stir-fried vegetables with tomatoes, peppers, and onions."

[1086] 4. Database

[1087] The database stores and manages the analyzed ingredient information, recording information such as the type, quantity, and freshness of ingredients.

[1088] 5. Food delivery

[1089] Based on the food ingredient information in the database, the server instructs the food delivery service's delivery staff on the optimal collection schedule and location. The delivery staff follows these instructions, collects the surplus ingredients at the specified date, time, and location, and delivers them to a partner cooking facility.

[1090] 6. Cooking facilities

[1091] The food preparation facility receives the collected ingredients, checks and cleans them, and then prepares them according to the recipes provided by the AI. The cooked food is then stored in an appropriate manner, such as refrigerated or frozen.

[1092] 7. Emotion Engine

[1093] The emotion engine installed on the user's device uses facial recognition technology and voice analysis to analyze the user's current emotional state. The analysis results are sent to a server, which then suggests optimal cooking menus based on the user's emotions.

[1094] Examples of concrete examples and prompts

[1095] As a specific example, a user takes a photo of two leftover tomatoes, one green pepper, and one onion at home and sends it to a server via a dedicated app. The server's generation AI analyzes the image and registers "two tomatoes, one green pepper, and one onion" in a database. A delivery person receives collection instructions, goes to the user's home to collect the ingredients, and transports them to a cooking facility. The generation AI generates a recipe for "stir-fried vegetables," which is then cooked at the cooking facility. Furthermore, the emotion engine on the user's device analyzes that the user is in a stressful state, and the server suggests a dish menu with a relaxing effect. The user then orders the suggested menu using a dedicated app, and a delivery person delivers the food from the cooking facility to the user's home.

[1096] An example of a prompt is as follows:

[1097] "I have two tomatoes, one bell pepper, and one onion. Can you suggest a dish I can make using these ingredients?"

[1098] "Please suggest a meal plan when the user is feeling stressed."

[1099] This allows for effective use of surplus ingredients and menu suggestions that meet the emotional needs of users.

[1100] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1101] Step 1:

[1102] Input: Images of surplus food taken by a user using a user device such as a smartphone.

[1103] Specific operation: The user launches a dedicated application and takes a photo of surplus ingredients (e.g., two tomatoes, one green pepper, and one onion).

[1104] Output: The captured image is saved in the user's device.

[1105] Next process: The saved image is sent to the server.

[1106] Step 2:

[1107] Input: Ingredient images sent from the user device.

[1108] Specific operation: The user taps the send button in the application to send the captured image to the server.

[1109] Output: The image data arrives at the server.

[1110] Next process: The server analyzes the received image.

[1111] Step 3:

[1112] Input: Image data received by the server.

[1113] Specific operation: The server inputs the received image into the generative AI model and uses image recognition technology to analyze the type, quantity, and freshness of ingredients. For example, the generative AI recognizes a red, round shape as a "tomato."

[1114] Output: The analysis result is "2 tomatoes, 1 bell pepper, and 1 onion."

[1115] Next process: The analysis results are registered in a database.

[1116] Step 4:

[1117] Input: Parsed ingredient information.

[1118] Specific operation: The server registers the analysis results in a database.

[1119] Output: Stored in the database as "2 tomatoes, 1 bell pepper, 1 onion".

[1120] Next action: Collection instructions are generated from the database.

[1121] Step 5:

[1122] Input: Ingredient information stored in the database.

[1123] Specific operation: Based on the ingredient information in the database, the server instructs the food delivery person on the optimal collection schedule and location. For example, it may instruct the person to "collect two tomatoes, one green pepper, and one onion from User A's address."

[1124] Output: Collection instructions received by the courier.

[1125] Next step: The delivery person collects the ingredients.

[1126] Step 6:

[1127] Input: Collection instructions received by the courier.

[1128] Specific operation: The delivery person will go to the user's address at the specified date and time and collect the excess ingredients.

[1129] Output: Recovered ingredients.

[1130] Next process: The recovered ingredients are transported to a food preparation facility.

[1131] Step 7:

[1132] Input: Recovered ingredients.

[1133] Specific operation: The delivery person transports the collected ingredients to a partner cooking facility.

[1134] Output: Food arriving at the cooking facility.

[1135] Next step: the ingredients are cooked.

[1136] Step 8:

[1137] Input: Ingredients arriving at the cooking facility.

[1138] Specific actions: The kitchen staff inspects the collected ingredients, cleans them, and then cooks them according to the recipe provided by the generative AI. For example, they make a vegetable stir-fry using tomatoes, peppers, and onions.

[1139] Output: Cooked food.

[1140] Next process: The dish is saved.

[1141] Step 9:

[1142] Input: Cooked food.

[1143] Specific Behavior: Store cooked food in the correct manner, such as by refrigerating or freezing.

[1144] Output: Saved dishes.

[1145] Next process: Emotion analysis on the user device.

[1146] Step 10:

[1147] Input: The user's emotional state.

[1148] Specific operation: The emotion engine installed on the user's device analyzes the user's emotional state using facial recognition and voice analysis.

[1149] Output: Parsed emotion information.

[1150] Next process: Menu suggestions are made based on emotional information.

[1151] Step 11:

[1152] Input: Parsed emotion information.

[1153] Specific operation: The server proposes the most suitable food menu based on the emotional information analyzed by the emotion engine. For example, if the user is feeling stressed, it will propose dishes that have a relaxing effect.

[1154] Output: A menu of dishes suggested to the user.

[1155] Next: The user orders food.

[1156] Step 12:

[1157] Input: Suggested food menu.

[1158] Specific operation: The user uses a dedicated application to select the desired dish from the menu and complete the order.

[1159] Output: Order information.

[1160] Next step: The ordered food is delivered.

[1161] Step 13:

[1162] Input: User's order information.

[1163] Specific operation: The server receives the order information and sends delivery instructions to the delivery person. The delivery person picks up the ordered food from the cooking facility and delivers it to the location specified by the user.

[1164] Output: The food delivered to the user.

[1165] Next process: The user's meal is completed and the system process ends.

[1166] (Application example 2)

[1167] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1168] This invention aims to address the issue of food waste while improving users' life satisfaction by suggesting optimal cooking menus based on the user's emotional state. Conventional food delivery services have had difficulty making effective use of surplus ingredients or providing personalized services based on the user's emotional state. This has led to increased food waste and a poor user experience.

[1169] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a generation AI means that analyzes the transmitted image and registers the ingredient information in a database, an emotion engine means that analyzes the user's emotional state using the user terminal, and a server means that suggests a dish menu based on the user's emotional state. This makes it possible to effectively use surplus ingredients and suggest a dish menu that is optimal for the user's current emotional state.

[1170] "Surplus food" refers to food that is left over in homes or facilities and that may be discarded without being used.

[1171] A "user terminal" is a device that a user uses to take pictures of surplus ingredients and input order information, and includes smartphones, tablets, and the like.

[1172] The "generative AI means" is an artificial intelligence technology that analyzes the transmitted image, identifies ingredient information, and registers it in a database.

[1173] The "emotion engine means" is a technology for analyzing the user's emotional state, and includes facial recognition technology, voice analysis technology, and the like.

[1174] The "server means" is a central processing unit that issues collection instructions and suggests cooking menus based on ingredient information and the user's emotional state.

[1175] "Delivery personnel means" refers to personnel and vehicles that actually collect surplus ingredients and deliver cooked meals to users.

[1176] "Affiliated Food Facility" means a food facility or restaurant with which we have contracted to prepare and store recovered food.

[1177] The "means for suggesting a cooking menu" is a technology that allows the generative AI to create the optimal cooking menu based on the user's emotional state and ingredient information in the database.

[1178] A "database" is an electronic data storage device for centrally managing analyzed ingredient information, user order information, and the like.

[1179] Overall system overview

[1180] This invention is a system that efficiently collects and manages surplus ingredients, and proposes and delivers meal plans tailored to the user's emotional state. The following describes in detail how this system is implemented.

[1181] Hardware and Software Use

[1182] This system mainly uses the following hardware and software:

[1183] Smartphone: Used to take photos of surplus ingredients and analyze user emotions

[1184] Server: Used for image analysis, food ingredient information registration using generative AI, user emotional state analysis, and menu suggestions

[1185] Generative AI model: Analyzes ingredient information and generates optimal cooking menus

[1186] Emotion engine: Analyzes the user's emotional state through facial recognition and voice analysis

[1187] Database: Stores information about ingredients and user orders

[1188] Explanation of program processing

[1189] 1. Photograph and send your surplus ingredients:

[1190] Users use a dedicated smartphone app to take a photo of leftover ingredients at home, and the app sends the image to a server.

[1191] 2. Image analysis and database registration:

[1192] The server uses a generative AI model to analyze the received images, identifying the type, quantity, and freshness of ingredients from the images and registering this information in a database.

[1193] 3. Instructions for collection and collection of ingredients:

[1194] The server generates an optimal collection schedule based on the food ingredient information registered in the database and sends instructions to the delivery staff, who then collect the ingredients at the specified date and time and transport them to the partner cooking facility.

[1195] 4. Create a food menu:

[1196] The generative AI model on the server creates the optimal recipe based on the ingredient information registered in the database. For example, it generates a recipe for "stir-fried vegetables with tomatoes, peppers, and onions."

[1197] 5. Cooking and storage:

[1198] Staff at partner kitchens will prepare the ingredients according to the recipes created by the generative AI, and the cooked food will be stored in an appropriate manner, such as refrigerated or frozen.

[1199] 6. User sentiment analysis:

[1200] The emotion engine on the user device analyzes the user's emotional state using a camera and microphone, and the analyzed emotion information is sent to the server.

[1201] 7. Menu suggestions based on emotion information:

[1202] The server uses a generative AI model to suggest optimal meal plans based on the analyzed emotional information. For example, if the user is feeling stressed, the server suggests meal plans with a relaxing effect.

[1203] 8. User Orders and Delivery:

[1204] The user places an order by selecting from the menu of suggested dishes through a dedicated app. The server receives the order information and sends delivery instructions to the delivery person. The delivery person then picks up the food from the cooking facility and delivers it to the user.

[1205] Specific examples

[1206] Specifically, User A takes a photo of two tomatoes, one green pepper, and one onion that they have left over at home and sends it to the server via the app. The server's generation AI analyzes this and registers "two tomatoes, one green pepper, and one onion" in a database. A delivery person then receives collection instructions, goes to User A's home to collect the ingredients, and transports them to a cooking facility. The generation AI suggests "stir-fried vegetables," and the dish is prepared at the cooking facility. When the emotion engine on the user's device analyzes that User A is feeling stressed, the server suggests "stir-fried vegetables with sesame flavor" as a dish that has a relaxing effect. Finally, User A orders this menu through the app, and a delivery person delivers it to their home. This makes it possible to make effective use of surplus ingredients, reduce food waste, and provide a service that is tailored to the user's emotions.

[1207] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1208] Step 1:

[1209] When a user realizes that they have surplus ingredients at home, they take a photo of the ingredients using a dedicated smartphone app. The input is the photo of the ingredients, and the output is image data stored on the smartphone. The smartphone stores this image data within the application.

[1210] Step 2:

[1211] A smartphone application sends a captured image to a server. The input is image data, and the output is the sent image data. Specifically, the application uploads the image file to the server using an HTTP request.

[1212] Step 3:

[1213] The server uses a generative AI model to analyze the transmitted image. The input is image data, and the output is analyzed ingredient information (type, quantity, freshness, etc.). The generative AI model analyzes the data using image recognition technology to identify ingredient information.

[1214] Step 4:

[1215] The server registers the analyzed ingredient information in a database. The input is the analyzed ingredient information, and the output is a record to be saved in the database. The server inserts the ingredient information into the database using an SQL query.

[1216] Step 5:

[1217] The server generates an optimal collection schedule based on the food ingredient information registered in the database and sends collection instructions to the delivery staff. The input is the food ingredient information in the database, and the output is the collection instructions sent to the delivery staff. The server uses a scheduling algorithm to determine the optimal collection date, time, and location.

[1218] Step 6:

[1219] The delivery person visits the user's address at the specified date and time to collect the excess ingredients. The input is the collection instruction from the server, and the output is the collected ingredients. The delivery person safely transports the ingredients using a container specially designed for food.

[1220] Step 7:

[1221] The delivery staff transports the collected ingredients to the partner cooking facility. The input is the collected ingredients, and the output is the ingredients delivered to the cooking facility. The delivery staff uses appropriate transportation methods to ensure the ingredients are kept fresh.

[1222] Step 8:

[1223] The generative AI model on the server creates the optimal cooking menu based on the ingredient information registered in the database. The input is the ingredient information in the database, and the output is the generated cooking menu. The generative AI model generates the optimal recipe by taking into account the combination of ingredients and cooking methods.

[1224] Step 9:

[1225] Partnered kitchens prepare ingredients according to recipes created by the generative AI. The input is the generated menu and ingredients, and the output is the cooked food. Kitchen staff follow the set procedures to create high-quality dishes.

[1226] Step 10:

[1227] The emotion engine on the user device analyzes the user's emotional state. The input is the user's facial image and voice data, and the output is analyzed emotional information. The emotion engine recognizes the user's emotions using facial recognition technology and voice analysis technology.

[1228] Step 11:

[1229] The server uses a generative AI model to propose optimal cooking menus based on the analyzed emotional information. The input is emotional information and ingredient information, and the output is an optimal cooking menu based on the emotions. The server proposes menus that have a relaxing effect or that are invigorating depending on the emotions.

[1230] Step 12:

[1231] Users place their orders by selecting from the proposed menu through a dedicated app. The input is the menu information, and the output is the user's order data. The app provides an interface that allows users to easily select menu items and complete their orders.

[1232] Step 13:

[1233] The server receives the user's order information and sends delivery instructions to the delivery person. The input is the user's order information and the output is delivery instructions. The server generates the optimal delivery route and schedule and sends the delivery instructions to the delivery person.

[1234] Step 14:

[1235] The delivery person receives the food from the cooking facility and delivers it to the location specified by the user. The input is the cooked food and delivery instructions, and the output is the food delivered to the user. The delivery person is required to deliver the ordered food safely and quickly.

[1236] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1237] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1238] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1239] [Fourth embodiment]

[1240] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1241] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1242] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1243] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1244] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1245] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1246] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1247] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1248] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1249] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1250] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1251] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1252] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1253] The system of the present invention provides a comprehensive solution to eliminate food waste through the efficient collection, management, cooking, and delivery of surplus food ingredients. The detailed process and specific embodiments are described below.

[1254] 1. Take a photo of your surplus food and send it to us

[1255] Users use a smartphone or other device to take a photo of surplus food items using a dedicated application. For example, they can take a photo of leftover tomatoes, peppers, onions, or other ingredients at home. After taking the photo, they can use the application's sending function to send the image to the server.

[1256] 2. Image analysis and database registration

[1257] The server uses a generation AI to analyze the received images. This generation AI uses image recognition technology to determine the type, size, quantity, and freshness of the ingredients. For example, it identifies a red, round object as a "tomato" and a green, elongated object as a "bell pepper." The analyzed ingredient information is then registered in a database.

[1258] 3. Instructions for collection and collection of ingredients

[1259] The server determines the optimal collection schedule and location based on the food ingredient information registered in the database. Collection instructions are sent to the food delivery person, informing them of the specific collection date, time, and location (e.g., User A's address). The delivery person visits the user's address at the specified date and time to collect the excess ingredients.

[1260] 4. Transporting ingredients and delivering them to cooking facilities

[1261] The delivery staff will then transport the collected ingredients to a partner cooking facility, where they will be inspected and cleaned.

[1262] 5. Create a food menu

[1263] The AI ​​on the server creates the optimal menu based on the ingredient information registered in the database. For example, it generates a recipe for "stir-fried vegetables with tomatoes, peppers, and onions." The generated menu and recipe are then sent to the cooking facility.

[1264] 6. Cooking and storage

[1265] The kitchen staff prepares the ingredients according to the recipes created by the generative AI, and once cooking is complete, the food is stored in the appropriate way, such as refrigerated or frozen.

[1266] 7. User Orders and Delivery

[1267] Users can order pre-cooked meals through a dedicated application. The application displays a list of available menu items, allowing users to select the desired dish and complete the order. The order information is sent to a server, which then sends delivery instructions to a delivery person. The delivery person collects the ordered food from the cooking facility and delivers it to the location specified by the user.

[1268] As a specific example, a user takes a photo of two tomatoes, one green pepper, and one onion that they have left over at home and sends it to the server via the app. The server's generation AI analyzes the photo and registers "two tomatoes, one green pepper, and one onion" in the database. A delivery person receives collection instructions, goes to the user's home to collect the ingredients, and transports them to a cooking facility. The generation AI suggests "stir-fried vegetables," and the dish is prepared and stored at the cooking facility. Finally, the user orders the menu item via the app, and the delivery person delivers it to their home. This makes effective use of surplus ingredients and effectively eliminates food waste.

[1269] The processing flow will be explained below.

[1270] Step 1:

[1271] The user launches the app and takes a photo of the surplus food items, such as tomatoes, peppers, or onions, using the camera function on their smartphone.

[1272] Step 2:

[1273] The device sends the captured image to the server. When the image data is sent, it is given a timestamp and user ID information.

[1274] Step 3:

[1275] The server receives the image and analyzes it using generative AI. Specifically, it uses an image recognition algorithm to determine the type, size, quantity, and freshness of the food. For example, it identifies a red, round object as a "tomato" and evaluates its freshness based on its size and color.

[1276] Step 4:

[1277] The server registers the analysis results in a database, recording detailed information such as the freshness of the ingredients (e.g., "2 tomatoes, 1 bell pepper, 1 onion") and the date and time the photo was taken.

[1278] Step 5:

[1279] The server determines the optimal collection schedule and location. Based on the information in the database, it sends collection instructions to the delivery person. These instructions include the date, time, and location of the food collection (e.g., User A's address).

[1280] Step 6:

[1281] The delivery person will collect the ingredients at the specified date, time and location. The delivery person will go to User A's address and pick up the surplus ingredients.

[1282] Step 7:

[1283] The delivery person transports the collected ingredients to the cooking facility. During the transport, the condition of the ingredients is easily checked and any problems are reported to the server.

[1284] Step 8:

[1285] The server communicates ingredient information to the cooking facility, including the type, quantity, and freshness of the ingredients.

[1286] Step 9:

[1287] The generative AI creates the optimal menu, generating recipes such as "stir-fried vegetables with tomatoes, peppers, and onions" based on the ingredients in the database, and sending them to the cooking facility.

[1288] Step 10:

[1289] The staff at the kitchen will prepare the food according to the recipes generated by the AI, and once the food is cooked, it will be stored appropriately, for example by refrigerating or freezing it.

[1290] Step 11:

[1291] Users order pre-cooked meals through a dedicated app, which displays available menu items, and users can select the desired dish to complete the order.

[1292] Step 12:

[1293] The server sends delivery instructions to the delivery person based on the order information. The delivery person is instructed to "Deliver the food from the cooking facility to User B's address."

[1294] Step 13:

[1295] The delivery person picks up the ordered food at the cooking facility and delivers it to the address of User B. After confirming that the food has been delivered safely, the delivery is reported to the server as completed.

[1296] Example 1

[1297] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1298] Currently, the effective use of surplus ingredients and the problem of food waste are social issues, but efficient systems for collection, management, cooking, and delivery are not fully established. There is also a lack of advanced analytical technology to determine the freshness of ingredients and propose appropriate cooking menus. For this reason, a system that can effectively utilize surplus ingredients is needed.

[1299] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1300] In this invention, the server includes a user terminal means for taking pictures of surplus ingredients, a means for sending the taken images to the server, and a generating AI means on the server for analyzing the sent images and registering ingredient information in a database, thereby enabling efficient collection, analysis, and management of surplus ingredients.

[1301] The server also includes a means for sending a collection instruction to a delivery person based on the ingredient information, and a delivery person means for collecting the ingredients according to the collection instruction and transporting them to the cooking facility, thereby enabling the ingredients to be collected and delivered to the cooking facility quickly.

[1302] Furthermore, the server includes a generation AI means for creating an optimal menu based on the ingredient information in the database, a means for cooking and storing ingredients at a cooking facility, a delivery staff means for delivering cooked food based on the user's order, a means for reporting the status of the food en route to the cooking facility to the server in real time, and a means for updating the server with information on completion of storage, thereby enabling the effective use of ingredients, appropriate cooking and storage, and prompt delivery to the user.

[1303] "Surplus ingredients" refers to ingredients or food that are left over and not consumed.

[1304] "User terminal means" refers to a device used by a user, such as a computer or smartphone, that has the function of photographing ingredients and transmitting information about them.

[1305] "Server" refers to a computer system that processes, stores, and transmits data over a network.

[1306] "Generative AI means" refers to technologies that use artificial intelligence (AI) to analyze data and generate specific functions or results.

[1307] "Delivery means" refers to the person or system that collects ingredients or food from a designated location and transports them to a designated location.

[1308] "Ingredient information" refers to data including details such as the type, quantity, and freshness of ingredients.

[1309] A "database" is a data collection point and a system for organizing, storing, and searching various information.

[1310] "Cooking facility" refers to a facility that receives, cooks, and preserves food ingredients.

[1311] "Food Menu" means a list of the types of food available and their details.

[1312] "Freshness" refers to the degree of freshness of food or ingredients, and is usually a standard for evaluating quality, expiration dates, etc.

[1313] "Preservation measures" refers to measures such as refrigeration, freezing, and packaging that are used to store cooked food and ingredients in an appropriate state.

[1314] The system of the present invention provides a comprehensive solution to eliminate food waste through the efficient collection, management, cooking, and delivery of surplus ingredients. Specific embodiments of the system of the present invention are described below.

[1315] Using a smartphone or other user device, a user can take a photo of surplus food items using a dedicated application. For example, they can take a photo of leftover tomatoes, peppers, onions, or other ingredients at home, and press the send button on the application to send the image to the server.

[1316] The server then launches a generative AI model to analyze the received images. This generative AI model uses image recognition technologies such as TensorFlow and PyTorch to determine the type, size, quantity, and freshness of the ingredients and registers them in a database. For example, it might identify a red, round object as a "tomato" and a green, elongated object as a "bell pepper."

[1317] Based on the food ingredient information registered in the database, the server determines the optimal collection schedule and location. Collection instructions are sent to the delivery person's device, and the delivery person visits the user's address at the specified date and time to collect the excess food. The GPS-equipped device provides the delivery person with the optimal route, supporting efficient collection.

[1318] The delivery person transports the collected ingredients to the cooking facility. During this process, the status of the delivery is reported to the server in real time. Upon arrival at the cooking facility, information confirming the arrival of the ingredients is sent to the server. The cooking facility staff checks the ingredients and cleans up if necessary.

[1319] The generative AI model on the server generates the optimal dish menu based on the ingredient information registered in the database. This process uses a natural language generation model (e.g., GPT-4). For example, a recipe for "stir-fried vegetables with tomatoes, peppers, and onions" is generated and sent to the cooking facility.

[1320] The staff at the kitchen prepares the ingredients according to the recipe created by the generative AI. Once the food is cooked, it is stored in an appropriate manner, such as refrigerated or frozen. This information about the completion of storage is updated to the server.

[1321] Users can order pre-cooked meals through a dedicated application. The application displays a list of available menu items, and users can select the desired dish to complete the order. The order information is sent to a server, which then sends delivery instructions to a delivery person. The delivery person collects the ordered food from the cooking facility and delivers it to the address specified by the user.

[1322] Examples:

[1323] The user takes a photo of two surplus tomatoes, one bell pepper, and one onion at home and sends it to the server via the app. The server's generation AI analyzes the photo and registers it in the database as "two tomatoes, one bell pepper, and one onion." A delivery person receives collection instructions, goes to the user's home to collect the ingredients, and transports them to a cooking facility. The generation AI suggests a recipe for "stir-fried vegetables," and the dish is prepared at the cooking facility based on this recipe and stored. Finally, the user orders the menu item via the app, and the delivery person delivers it to their home. This makes effective use of surplus ingredients and effectively eliminates food waste.

[1324] Example prompt for a generative AI model:

[1325] "Please suggest the best recipe based on the ingredient information received from the user. For example, if two tomatoes, one bell pepper, and one onion are provided, please consider what kind of dish can be made using them and provide a detailed recipe."

[1326] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1327] Step 1:

[1328] The user uses a dedicated smartphone application to take a photo of surplus ingredients. For example, they can take a photo of leftover tomatoes, peppers, onions, and other ingredients at home. The input is an image of each ingredient, and the output is an image file.

[1329] Step 2:

[1330] The user presses the send button in the dedicated application to send the captured image to the server. The input is the image file obtained in step 1, and the output is a notification that the image file has been uploaded to the server.

[1331] Step 3:

[1332] The server launches a generative AI model to analyze the received image. The input is the transmitted image file, and the generative AI model performs image recognition using TensorFlow or PyTorch. The output is the recognized ingredient information (e.g., 2 tomatoes, 1 bell pepper, 1 onion). The server registers this information in a database.

[1333] Step 4:

[1334] The server determines the optimal collection schedule and location based on the food ingredient information registered in the database. The input is the food ingredient information registered in the database, and the output is collection instructions to the delivery person. The collection instructions include the specific collection date, time, and location.

[1335] Step 5:

[1336] The delivery person visits the user's address at the specified date and time to collect the excess ingredients. The input is the collection instruction from the server, and the output is the collected ingredients. The delivery person's device is equipped with a GPS, which provides the optimal route.

[1337] Step 6:

[1338] The delivery person transports the collected ingredients to the cooking facility and reports the status of the delivery to the server in real time. The input is the collected ingredients, and the output is a notification of arrival at the cooking facility.

[1339] Step 7:

[1340] The kitchen staff checks the food as it arrives and cleans it up as necessary. The input is the delivered food, and the output is the cleaned food.

[1341] Step 8:

[1342] A generative AI model on the server generates optimal cooking menus based on ingredient information registered in a database. The input is ingredient information from the database, and the generative AI model uses natural language generation technologies such as GPT-4. The output is the generated cooking menu and recipe.

[1343] Step 9:

[1344] The kitchen staff prepares the ingredients according to the recipes created by the generative AI model. The input is the generated food menu and recipe, and the output is the cooked food.

[1345] Step 10:

[1346] Once the food is cooked, it is stored in an appropriate way, such as refrigerated or frozen. The input is the cooked food, and the output is the stored food. This storage completion information is updated to the server.

[1347] Step 11:

[1348] A user orders cooked food using a dedicated application. The input is the user's order information, and the output is a notification that the order information has been sent to the server. The application displays a list of available menu items.

[1349] Step 12:

[1350] The server sends the order information to the delivery person as delivery instructions. The input is the user's order information, and the output is delivery instructions to the delivery person.

[1351] Step 13:

[1352] The delivery person receives the ordered food from the food preparation facility and delivers it to the address specified by the user. The input is the delivery instruction and the cooked food, and the output is a notification to the user that the food has been delivered.

[1353] (Application example 1)

[1354] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1355] A comprehensive system is needed to effectively eliminate food waste by efficiently collecting, managing, cooking, and delivering surplus ingredients. It is also required to generate optimal menus based on ingredient information, allowing users to easily order. However, current systems lack sufficient information on ingredient freshness and generate optimal collection schedules, making efficient operation difficult.

[1356] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1357] In this invention, the server includes a means for taking a photo of surplus ingredients using a user terminal and sending the image, a generating AI model means for analyzing the sent image and registering ingredient information in a database, and a means for generating food delivery instructions based on the ingredient information and sending them to a delivery person, thereby enabling the efficient collection, management, cooking, and delivery of surplus ingredients.

[1358] A "user terminal" is a device used to take pictures of surplus ingredients and send the images. This includes smartphones.

[1359] "Image analysis" is the process of analyzing the transmitted image and extracting ingredient information.

[1360] A "generative AI model" is a system that uses machine learning algorithms to create cooking menus based on image analysis and ingredient information.

[1361] A "database" is an information system that registers and manages analyzed ingredient information.

[1362] "Food delivery" is a service that delivers food and ingredients to a location specified by the user.

[1363] "Delivery Instructions" are instructions that indicate the specific schedule and location for collecting and delivering ingredients and food.

[1364] "Cooking facilities" are facilities for cooking and storing collected food ingredients.

[1365] A "cooking menu" is a recipe or list of dishes created based on ingredient information in the database.

[1366] "User order" is an operation performed by a user to order a cooked dish.

[1367] A "smartphone application" is an app that allows users to check and order information about ingredients and menu items.

[1368] This invention is a comprehensive system for eliminating food waste through the efficient collection, management, cooking, and delivery of surplus food ingredients. The system is composed of the following elements:

[1369] 1. User Device:

[1370] Users take photos of leftover ingredients using a user device such as a smartphone. The images are then sent to a server via a dedicated application. For example, users can take photos of leftover ingredients such as tomatoes, peppers, and onions at home.

[1371] 2. Image analysis and database registration:

[1372] The server uses a generative AI model to analyze the received images. This generative AI model uses image recognition technology to determine the type, size, quantity, and freshness of ingredients. For example, it might identify a red, round object as a "tomato" and a green, elongated object as a "bell pepper." The analyzed ingredient information is then registered in a database.

[1373] 3. Ingredient collection and scheduling:

[1374] The server determines the optimal collection schedule and location based on the food ingredient information registered in the database. Collection instructions are sent to the food delivery person, informing them of the specific collection date, time, and location (e.g., the user's address). The delivery person visits the user's address at the specified date and time to collect the excess ingredients.

[1375] 4. Transportation and delivery of ingredients to cooking facilities:

[1376] The delivery staff will then transport the collected ingredients to a partner cooking facility, where they will be inspected and cleaned.

[1377] 5. Create a food menu:

[1378] The generative AI model on the server creates the optimal cooking menu based on the ingredient information registered in the database. For example, it generates a recipe for "stir-fried vegetables with tomatoes, peppers, and onions." The generated menu and recipe are sent to the cooking facility.

[1379] 6. Preparation and storage:

[1380] The kitchen staff prepares the ingredients according to the recipes created by the generative AI, and once cooking is complete, the food is stored in the appropriate way, such as refrigerated or frozen.

[1381] 7. User Orders and Delivery:

[1382] Users can order pre-cooked meals through a smartphone application. The application displays a list of available menu items, and users can select the desired dish to complete the order. The order information is sent to a server, which then sends delivery instructions to a delivery person. The delivery person collects the ordered food from the cooking facility and delivers it to the location specified by the user.

[1383] Hardware and software used:

[1384] Smartphone: Used by users to take pictures of ingredients and operate applications.

[1385] Server: Performs central processing such as image analysis, database management, and delivery instruction generation.

[1386] Generative AI models: Includes image recognition technologies such as Google Cloud Vision API.

[1387] Cooking facilities: Facilities for cooking and storing recovered food.

[1388] Examples:

[1389] For example, a user takes a photo of two tomatoes, one green pepper, and one onion that they have left over at home and sends it to the server via the app. The generative AI model on the server analyzes the image and registers "two tomatoes, one green pepper, and one onion" in the database. A delivery person receives collection instructions, goes to the user's home to collect the ingredients, and transports them to a cooking facility. The generative AI model suggests "stir-fried vegetables," and the dish is prepared and stored at the cooking facility. Finally, the user orders the menu item via the app, and the delivery person delivers it to their home. This makes effective use of surplus ingredients and effectively eliminates food waste.

[1390] Example prompt sentence:

[1391] Image analysis prompt:

[1392] "Take a picture of leftover food in your home and analyze this image. Identify specific ingredients like tomatoes, peppers, and onions."

[1393] Recipe generation prompt:

[1394] "Generate the best recipe using the following ingredients: 2 tomatoes, 1 bell pepper, and 1 onion."

[1395] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1396] Step 1:

[1397] The user takes a photo of the surplus ingredients using a device such as a smartphone. For example, the user takes a photo of a tomato, pepper, or onion. This provides the image of the ingredients as input data.

[1398] Step 2:

[1399] The captured image is sent to the server. The user uses a dedicated application on their device to send the captured image of the ingredients to the server. The input data is the image of the ingredients, and processing begins when the server receives it.

[1400] Step 3:

[1401] The server analyzes the image. The server uses a generative AI model (such as Google Cloud Vision API) to analyze the received image. Information such as the type, size, quantity, and freshness of the ingredients is extracted from the image, and ingredient information is obtained from the input image data.

[1402] Step 4:

[1403] The ingredient information is registered in a database. The server registers the analyzed ingredient information in the database. Information such as the type and quantity of ingredients is saved in the database. The input is the analyzed ingredient information, and the output is the registration result in the database.

[1404] Step 5:

[1405] Generates an optimal collection schedule. The server determines the collection date, time, and location based on the food ingredient information registered in the database. The input data is the food ingredient information in the database, and the server generates an optimal collection schedule and notifies the delivery person as output.

[1406] Step 6:

[1407] The delivery person collects the ingredients. The delivery person follows instructions from the server and goes to the specified date, time and location to collect the ingredients. This allows the collected ingredients to be obtained from the user. The input is the server's collection instructions, and the output is the collected ingredients.

[1408] Step 7:

[1409] The collected ingredients are transported to the cooking facility. The delivery person transports the collected ingredients to the partner cooking facility. The input is the collected ingredients, and the output is the completion of transportation to the cooking facility.

[1410] Step 8:

[1411] The server generates a cooking menu. The generative AI model on the server creates the optimal cooking menu based on the ingredient information registered in the database. The input data is the ingredient information in the database, and the output is the generated cooking menu.

[1412] Step 9:

[1413] The food is cooked and stored in the kitchen. The staff at the kitchen follow the recipe generated by the server to prepare the food. After cooking, the food is stored by refrigeration or freezing. The input is the generated recipe, and the output is the cooked food.

[1414] Step 10:

[1415] The server accepts orders from users and issues delivery instructions. Users order pre-cooked food through a smartphone application, and the server issues instructions to the delivery person. The input is the user's order information, and the output is delivery instructions.

[1416] Step 11:

[1417] The delivery person delivers the ordered food to the user. The delivery person receives the food from the cooking facility and delivers it to the location specified by the user. The input is the delivery instruction and the ordered food, and the output is the completion of the food delivery.

[1418] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1419] The system of the present invention not only eliminates food waste through the efficient collection, management, cooking, and delivery of surplus ingredients, but also enables the system to suggest optimal cooking menus based on the user's emotional state. The system includes a user terminal, a server, a generation AI, a database, food delivery, cooking facilities, and an emotion engine. The detailed process and specific embodiments are described below.

[1420] 1. Take a photo of your surplus food and send it to us

[1421] Users use a smartphone or other device to take a photo of surplus food items using a dedicated application. For example, they can take a photo of leftover tomatoes, peppers, onions, or other ingredients at home. After taking the photo, they can use the application's sending function to send the image to the server.

[1422] 2. Image analysis and database registration

[1423] The server uses a generation AI to analyze the received images. This generation AI uses image recognition technology to determine the type, size, quantity, and freshness of the ingredients. For example, it identifies a red, round object as a "tomato" and a green, elongated object as a "bell pepper." The analyzed ingredient information is then registered in a database.

[1424] 3. Instructions for collection and collection of ingredients

[1425] The server determines the optimal collection schedule and location based on the food ingredient information registered in the database. Collection instructions are sent to the food delivery person, informing them of the specific collection date, time, and location (e.g., User A's address). The delivery person visits the user's address at the specified date and time to collect the excess ingredients.

[1426] 4. Transporting ingredients and delivering them to cooking facilities

[1427] The delivery staff will then transport the collected ingredients to a partner cooking facility, where they will be inspected and cleaned.

[1428] 5. Create a food menu

[1429] The AI ​​on the server creates the optimal menu based on the ingredient information registered in the database. For example, it generates a recipe for "stir-fried vegetables with tomatoes, peppers, and onions." The generated menu and recipe are then sent to the cooking facility.

[1430] 6. Cooking and storage

[1431] The kitchen staff prepares the ingredients according to the recipes created by the generative AI, and once cooking is complete, the food is stored in the appropriate way, such as refrigerated or frozen.

[1432] 7. User Sentiment Analysis

[1433] The emotion engine installed in the user device analyzes the user's emotions. For example, it uses facial recognition technology and voice analysis to recognize the user's current emotions (happiness, sadness, stress, etc.).

[1434] 8. Menu suggestions based on emotion information

[1435] The server proposes optimal meal menus based on the user's emotional information analyzed by the emotion engine. For example, if the user is feeling stressed, it will propose meal menus using ingredients that have a relaxing effect. The meal menus are also displayed on the user's device, expanding the user's options.

[1436] 9. User Orders and Delivery

[1437] Users can order pre-cooked meals through a dedicated application. The application displays a list of available menu items, allowing users to select the desired dish and complete the order. The order information is sent to a server, which then sends delivery instructions to a delivery person. The delivery person collects the ordered food from the cooking facility and delivers it to the location specified by the user.

[1438] As a specific example, a user takes a photo of two surplus tomatoes, one bell pepper, and one onion at home and sends it to the server via the app. The server's generation AI analyzes the image and registers "two tomatoes, one bell pepper, and one onion" in a database. A delivery person receives collection instructions, goes to the user's home to collect the ingredients, and transports them to a cooking facility. The generation AI suggests "stir-fried vegetables," and the dish is prepared and stored at the cooking facility. If the emotion engine on the user's device analyzes that the user is feeling stressed, the server suggests a dish with a relaxing effect. Finally, the user orders the menu item via the app, and the delivery person delivers it to their home. This makes effective use of surplus ingredients, effectively eliminating food waste, and also meets the user's emotional needs.

[1439] The processing flow will be explained below.

[1440] Step 1:

[1441] The user launches the app and takes a photo of the surplus food items, such as tomatoes, peppers, or onions, using the camera function on their smartphone.

[1442] Step 2:

[1443] The device sends the captured image to the server. When the image data is sent, it is given a timestamp and user ID information.

[1444] Step 3:

[1445] The server receives the image and analyzes it using generative AI. Specifically, it uses an image recognition algorithm to determine the type, size, quantity, and freshness of the food. For example, it identifies a red, round object as a "tomato" and evaluates its freshness based on its size and color.

[1446] Step 4:

[1447] The server registers the analysis results in a database. Along with ingredient information (e.g., "2 tomatoes, 1 bell pepper, 1 onion"), detailed information such as freshness and the date and time the photo was taken is also recorded. User emotional information is also recorded here.

[1448] Step 5:

[1449] The device activates an emotion engine to analyze the user's emotions, for example, by using voice analysis and facial recognition technology to recognize the user's current emotions (happiness, sadness, stress, etc.).

[1450] Step 6:

[1451] The server determines the optimal collection schedule and location. Based on the information in the database, it sends collection instructions to the delivery person. These instructions include the date, time, and location of the food collection (e.g., User A's address).

[1452] Step 7:

[1453] The delivery person will collect the ingredients at the specified date, time and location. The delivery person will go to User A's address and pick up the surplus ingredients.

[1454] Step 8:

[1455] The delivery person transports the collected ingredients to the cooking facility. During the transport, the condition of the ingredients is easily checked and any problems are reported to the server.

[1456] Step 9:

[1457] The server communicates ingredient information to the cooking facility, including the type, quantity, and freshness of the ingredients.

[1458] Step 10:

[1459] The generative AI creates the optimal cooking menu. Based on the ingredient information in the database and the user's emotional information, it generates a recipe such as "Stir-fried vegetables with tomatoes, peppers, and onions" and sends it to the cooking facility.

[1460] Step 11:

[1461] The staff at the kitchen will prepare the food according to the recipes generated by the AI, and once the food is cooked, it will be stored appropriately, for example by refrigerating or freezing it.

[1462] Step 12:

[1463] Users order pre-cooked meals through a dedicated app, which displays available menu items, and users can select the desired dish to complete the order.

[1464] Step 13:

[1465] The server sends delivery instructions to the delivery person based on the order information. The delivery person is instructed to "Deliver the food from the cooking facility to User B's address."

[1466] Step 14:

[1467] The delivery person picks up the ordered food at the cooking facility and delivers it to the address of User B. After confirming that the food has been delivered safely, the delivery is reported to the server as completed.

[1468] As a specific example, a user takes a photo of two surplus tomatoes, one bell pepper, and one onion at home and sends it to a server via the app. The server's generation AI analyzes the image and registers "two tomatoes, one bell pepper, and one onion" in a database. The device's emotion engine analyzes the user's stress and registers this information in the database. A delivery person receives collection instructions, goes to the user's home, collects the ingredients, and transports them to a cooking facility. The generation AI suggests a "relaxing stir-fry vegetable dish" based on the user's emotions, and the dish is prepared and stored at the cooking facility. Finally, the user orders the menu item via the app, and the delivery person delivers it to their home. This makes effective use of surplus ingredients, effectively eliminating food waste, and also meets the user's emotional needs.

[1469] Example 2

[1470] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1471] In modern society, excessive food purchases and improper storage result in large amounts of surplus food, making food waste a serious problem. Furthermore, since it is difficult to provide meals that meet the diverse emotional needs of users, appropriate measures are required to improve user satisfaction. To solve these issues, an efficient and optimal system is required.

[1472] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1473] In this invention, the server includes a user terminal means for taking pictures of surplus ingredients, a generating AI means for analyzing the transmitted images and registering ingredient information in a database, and a means for proposing optimal recipes based on emotional information, thereby enabling effective use of surplus ingredients and proposing menus that meet the emotional needs of users.

[1474] "Surplus food" refers to food that is left unused in homes, restaurants, etc.

[1475] "User terminal" refers to a portable electronic device such as a smartphone or tablet used by a user.

[1476] "Generative AI" refers to artificial intelligence that uses machine learning algorithms to analyze data and perform specific tasks automatically.

[1477] A "database" refers to a system for efficiently storing, managing, and retrieving data.

[1478] "Food delivery" refers to a service that delivers pre-cooked food to the customer's home.

[1479] "Delivery person" refers to a person whose job is to deliver goods or food from a designated location to another.

[1480] "Cooking facility" means the place where food is prepared and stored.

[1481] "Emotion analysis" refers to the process of measuring and analyzing a user's emotional state using data such as facial expressions and voice.

[1482] "Cooking Menu" refers to a recipe or list of dishes prepared using specific ingredients.

[1483] "Proposal means" refers to techniques and methods that present optimal options based on the user's requests and circumstances.

[1484] This invention is a system that efficiently collects, analyzes, manages, cooks, and delivers surplus ingredients. It also has a function that suggests optimal cooking menus based on the user's emotional state. This system consists of a user terminal, a server, a generation AI, a database, food delivery, cooking facilities, and an emotion engine.

[1485] System configuration

[1486] 1. User Device

[1487] The user takes a photo of the surplus ingredients using a user device such as a smartphone or tablet, launches a dedicated application, takes a photo of the ingredients, and sends it to the server.

[1488] 2. Server

[1489] The server uses a generative AI model to analyze the received images. This generative AI model uses image recognition technology to determine the type, quantity, and freshness of ingredients. The analysis results are then registered in a database.

[1490] 3. Generative AI Models

[1491] The generative AI creates optimal recipes based on the collected information on surplus ingredients. For example, it can generate a recipe for "stir-fried vegetables with tomatoes, peppers, and onions."

[1492] 4. Database

[1493] The database stores and manages the analyzed ingredient information, recording information such as the type, quantity, and freshness of ingredients.

[1494] 5. Food delivery

[1495] Based on the food ingredient information in the database, the server instructs the food delivery service's delivery staff on the optimal collection schedule and location. The delivery staff follows these instructions, collects the surplus ingredients at the specified date, time, and location, and delivers them to a partner cooking facility.

[1496] 6. Cooking facilities

[1497] The food preparation facility receives the collected ingredients, checks and cleans them, and then prepares them according to the recipes provided by the AI. The cooked food is then stored in an appropriate manner, such as refrigerated or frozen.

[1498] 7. Emotion Engine

[1499] The emotion engine installed on the user's device uses facial recognition technology and voice analysis to analyze the user's current emotional state. The analysis results are sent to a server, which then suggests optimal cooking menus based on the user's emotions.

[1500] Examples of concrete examples and prompts

[1501] As a specific example, a user takes a photo of two leftover tomatoes, one green pepper, and one onion at home and sends it to a server via a dedicated app. The server's generation AI analyzes the image and registers "two tomatoes, one green pepper, and one onion" in a database. A delivery person receives collection instructions, goes to the user's home to collect the ingredients, and transports them to a cooking facility. The generation AI generates a recipe for "stir-fried vegetables," which is then cooked at the cooking facility. Furthermore, the emotion engine on the user's device analyzes that the user is in a stressful state, and the server suggests a dish menu with a relaxing effect. The user then orders the suggested menu using a dedicated app, and a delivery person delivers the food from the cooking facility to the user's home.

[1502] An example of a prompt is as follows:

[1503] "I have two tomatoes, one bell pepper, and one onion. Can you suggest a dish I can make using these ingredients?"

[1504] "Please suggest a meal plan when the user is feeling stressed."

[1505] This allows for effective use of surplus ingredients and menu suggestions that meet the emotional needs of users.

[1506] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1507] Step 1:

[1508] Input: Images of surplus food taken by a user using a user device such as a smartphone.

[1509] Specific operation: The user launches a dedicated application and takes a photo of surplus ingredients (e.g., two tomatoes, one green pepper, and one onion).

[1510] Output: The captured image is saved in the user's device.

[1511] Next process: The saved image is sent to the server.

[1512] Step 2:

[1513] Input: Ingredient images sent from the user device.

[1514] Specific operation: The user taps the send button in the application to send the captured image to the server.

[1515] Output: The image data arrives at the server.

[1516] Next process: The server analyzes the received image.

[1517] Step 3:

[1518] Input: Image data received by the server.

[1519] Specific operation: The server inputs the received image into the generative AI model and uses image recognition technology to analyze the type, quantity, and freshness of ingredients. For example, the generative AI recognizes a red, round shape as a "tomato."

[1520] Output: The analysis result is "2 tomatoes, 1 bell pepper, and 1 onion."

[1521] Next process: The analysis results are registered in a database.

[1522] Step 4:

[1523] Input: Parsed ingredient information.

[1524] Specific operation: The server registers the analysis results in a database.

[1525] Output: Stored in the database as "2 tomatoes, 1 bell pepper, 1 onion".

[1526] Next action: Collection instructions are generated from the database.

[1527] Step 5:

[1528] Input: Ingredient information stored in the database.

[1529] Specific operation: Based on the ingredient information in the database, the server instructs the food delivery person on the optimal collection schedule and location. For example, it may instruct the person to "collect two tomatoes, one green pepper, and one onion from User A's address."

[1530] Output: Collection instructions received by the courier.

[1531] Next step: The delivery person collects the ingredients.

[1532] Step 6:

[1533] Input: Collection instructions received by the courier.

[1534] Specific operation: The delivery person will go to the user's address at the specified date and time and collect the excess ingredients.

[1535] Output: Recovered ingredients.

[1536] Next process: The recovered ingredients are transported to a food preparation facility.

[1537] Step 7:

[1538] Input: Recovered ingredients.

[1539] Specific operation: The delivery person transports the collected ingredients to a partner cooking facility.

[1540] Output: Food arriving at the cooking facility.

[1541] Next step: the ingredients are cooked.

[1542] Step 8:

[1543] Input: Ingredients arriving at the cooking facility.

[1544] Specific actions: The kitchen staff inspects the collected ingredients, cleans them, and then cooks them according to the recipe provided by the generative AI. For example, they make a vegetable stir-fry using tomatoes, peppers, and onions.

[1545] Output: Cooked food.

[1546] Next process: The dish is saved.

[1547] Step 9:

[1548] Input: Cooked food.

[1549] Specific Behavior: Store cooked food in the correct manner, such as by refrigerating or freezing.

[1550] Output: Saved dishes.

[1551] Next process: Emotion analysis on the user device.

[1552] Step 10:

[1553] Input: The user's emotional state.

[1554] Specific operation: The emotion engine installed on the user's device analyzes the user's emotional state using facial recognition and voice analysis.

[1555] Output: Parsed emotion information.

[1556] Next process: Menu suggestions are made based on emotional information.

[1557] Step 11:

[1558] Input: Parsed emotion information.

[1559] Specific operation: The server proposes the most suitable food menu based on the emotional information analyzed by the emotion engine. For example, if the user is feeling stressed, it will propose dishes that have a relaxing effect.

[1560] Output: A menu of dishes suggested to the user.

[1561] Next: The user orders food.

[1562] Step 12:

[1563] Input: Suggested food menu.

[1564] Specific operation: The user uses a dedicated application to select the desired dish from the menu and complete the order.

[1565] Output: Order information.

[1566] Next step: The ordered food is delivered.

[1567] Step 13:

[1568] Input: User's order information.

[1569] Specific operation: The server receives the order information and sends delivery instructions to the delivery person. The delivery person picks up the ordered food from the cooking facility and delivers it to the location specified by the user.

[1570] Output: The food delivered to the user.

[1571] Next process: The user's meal is completed and the system process ends.

[1572] (Application example 2)

[1573] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1574] This invention aims to address the issue of food waste while improving users' life satisfaction by suggesting optimal cooking menus based on the user's emotional state. Conventional food delivery services have had difficulty making effective use of surplus ingredients or providing personalized services based on the user's emotional state. This has led to increased food waste and a poor user experience.

[1575] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a generation AI means that analyzes the transmitted image and registers the ingredient information in a database, an emotion engine means that analyzes the user's emotional state using the user terminal, and a server means that suggests a dish menu based on the user's emotional state. This makes it possible to effectively use surplus ingredients and suggest a dish menu that is optimal for the user's current emotional state.

[1576] "Surplus food" refers to food that is left over in homes or facilities and that may be discarded without being used.

[1577] A "user terminal" is a device that a user uses to take pictures of surplus ingredients and input order information, and includes smartphones, tablets, and the like.

[1578] The "generative AI means" is an artificial intelligence technology that analyzes the transmitted image, identifies ingredient information, and registers it in a database.

[1579] The "emotion engine means" is a technology for analyzing the user's emotional state, and includes facial recognition technology, voice analysis technology, and the like.

[1580] The "server means" is a central processing unit that issues collection instructions and suggests cooking menus based on ingredient information and the user's emotional state.

[1581] "Delivery personnel means" refers to personnel and vehicles that actually collect surplus ingredients and deliver cooked meals to users.

[1582] "Affiliated Food Facility" means a food facility or restaurant with which we have contracted to prepare and store recovered food.

[1583] The "means for suggesting a cooking menu" is a technology that allows the generative AI to create the optimal cooking menu based on the user's emotional state and ingredient information in the database.

[1584] A "database" is an electronic data storage device for centrally managing analyzed ingredient information, user order information, and the like.

[1585] Overall system overview

[1586] This invention is a system that efficiently collects and manages surplus ingredients, and proposes and delivers meal plans tailored to the user's emotional state. The following describes in detail how this system is implemented.

[1587] Hardware and Software Use

[1588] This system mainly uses the following hardware and software:

[1589] Smartphone: Used to take photos of surplus ingredients and analyze user emotions

[1590] Server: Used for image analysis, food ingredient information registration using generative AI, user emotional state analysis, and menu suggestions

[1591] Generative AI model: Analyzes ingredient information and generates optimal cooking menus

[1592] Emotion engine: Analyzes the user's emotional state through facial recognition and voice analysis

[1593] Database: Stores information about ingredients and user orders

[1594] Explanation of program processing

[1595] 1. Photograph and send your surplus ingredients:

[1596] Users use a dedicated smartphone app to take a photo of leftover ingredients at home, and the app sends the image to a server.

[1597] 2. Image analysis and database registration:

[1598] The server uses a generative AI model to analyze the received images, identifying the type, quantity, and freshness of ingredients from the images and registering this information in a database.

[1599] 3. Instructions for collection and collection of ingredients:

[1600] The server generates an optimal collection schedule based on the food ingredient information registered in the database and sends instructions to the delivery staff, who then collect the ingredients at the specified date and time and transport them to the partner cooking facility.

[1601] 4. Create a food menu:

[1602] The generative AI model on the server creates the optimal recipe based on the ingredient information registered in the database. For example, it generates a recipe for "stir-fried vegetables with tomatoes, peppers, and onions."

[1603] 5. Cooking and storage:

[1604] Staff at partner kitchens will prepare the ingredients according to the recipes created by the generative AI, and the cooked food will be stored in an appropriate manner, such as refrigerated or frozen.

[1605] 6. User sentiment analysis:

[1606] The emotion engine on the user device analyzes the user's emotional state using a camera and microphone, and the analyzed emotion information is sent to the server.

[1607] 7. Menu suggestions based on emotion information:

[1608] The server uses a generative AI model to suggest optimal meal plans based on the analyzed emotional information. For example, if the user is feeling stressed, the server suggests meal plans with a relaxing effect.

[1609] 8. User Orders and Delivery:

[1610] The user places an order by selecting from the menu of suggested dishes through a dedicated app. The server receives the order information and sends delivery instructions to the delivery person. The delivery person then picks up the food from the cooking facility and delivers it to the user.

[1611] Specific examples

[1612] Specifically, User A takes a photo of two tomatoes, one green pepper, and one onion that they have left over at home and sends it to the server via the app. The server's generation AI analyzes this and registers "two tomatoes, one green pepper, and one onion" in a database. A delivery person then receives collection instructions, goes to User A's home to collect the ingredients, and transports them to a cooking facility. The generation AI suggests "stir-fried vegetables," and the dish is prepared at the cooking facility. When the emotion engine on the user's device analyzes that User A is feeling stressed, the server suggests "stir-fried vegetables with sesame flavor" as a dish that has a relaxing effect. Finally, User A orders this menu through the app, and a delivery person delivers it to their home. This makes it possible to make effective use of surplus ingredients, reduce food waste, and provide a service that is tailored to the user's emotions.

[1613] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1614] Step 1:

[1615] When a user realizes that they have surplus ingredients at home, they take a photo of the ingredients using a dedicated smartphone app. The input is the photo of the ingredients, and the output is image data stored on the smartphone. The smartphone stores this image data within the application.

[1616] Step 2:

[1617] A smartphone application sends a captured image to a server. The input is image data, and the output is the sent image data. Specifically, the application uploads the image file to the server using an HTTP request.

[1618] Step 3:

[1619] The server uses a generative AI model to analyze the transmitted image. The input is image data, and the output is analyzed ingredient information (type, quantity, freshness, etc.). The generative AI model analyzes the data using image recognition technology to identify ingredient information.

[1620] Step 4:

[1621] The server registers the analyzed ingredient information in a database. The input is the analyzed ingredient information, and the output is a record to be saved in the database. The server inserts the ingredient information into the database using an SQL query.

[1622] Step 5:

[1623] The server generates an optimal collection schedule based on the food ingredient information registered in the database and sends collection instructions to the delivery staff. The input is the food ingredient information in the database, and the output is the collection instructions sent to the delivery staff. The server uses a scheduling algorithm to determine the optimal collection date, time, and location.

[1624] Step 6:

[1625] The delivery person visits the user's address at the specified date and time to collect the excess ingredients. The input is the collection instruction from the server, and the output is the collected ingredients. The delivery person safely transports the ingredients using a container specially designed for food.

[1626] Step 7:

[1627] The delivery staff transports the collected ingredients to the partner cooking facility. The input is the collected ingredients, and the output is the ingredients delivered to the cooking facility. The delivery staff uses appropriate transportation methods to ensure the ingredients are kept fresh.

[1628] Step 8:

[1629] The generative AI model on the server creates the optimal cooking menu based on the ingredient information registered in the database. The input is the ingredient information in the database, and the output is the generated cooking menu. The generative AI model generates the optimal recipe by taking into account the combination of ingredients and cooking methods.

[1630] Step 9:

[1631] Partnered kitchens prepare ingredients according to recipes created by the generative AI. The input is the generated menu and ingredients, and the output is the cooked food. Kitchen staff follow the set procedures to create high-quality dishes.

[1632] Step 10:

[1633] The emotion engine on the user device analyzes the user's emotional state. The input is the user's facial image and voice data, and the output is analyzed emotional information. The emotion engine recognizes the user's emotions using facial recognition technology and voice analysis technology.

[1634] Step 11:

[1635] The server uses a generative AI model to propose optimal cooking menus based on the analyzed emotional information. The input is emotional information and ingredient information, and the output is an optimal cooking menu based on the emotions. The server proposes menus that have a relaxing effect or that are invigorating depending on the emotions.

[1636] Step 12:

[1637] Users place their orders by selecting from the proposed menu through a dedicated app. The input is the menu information, and the output is the user's order data. The app provides an interface that allows users to easily select menu items and complete their orders.

[1638] Step 13:

[1639] The server receives the user's order information and sends delivery instructions to the delivery person. The input is the user's order information and the output is delivery instructions. The server generates the optimal delivery route and schedule and sends the delivery instructions to the delivery person.

[1640] Step 14:

[1641] The delivery person receives the food from the cooking facility and delivers it to the location specified by the user. The input is the cooked food and delivery instructions, and the output is the food delivered to the user. The delivery person is required to deliver the ordered food safely and quickly.

[1642] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1643] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1644] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1645] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1646] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1647] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1648] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1649] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1650] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1651] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1652] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1653] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1654] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1655] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1656] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1657] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1658] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1659] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1660] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1661] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1662] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1663] The following is further disclosed regarding the above embodiment.

[1664] (Claim 1)

[1665] a user terminal for taking a photo of surplus ingredients;

[1666] means for transmitting the captured image to a server;

[1667] A server-based AI means for analyzing the transmitted images and registering the food information in a database;

[1668] a server means for transmitting a collection instruction to a food delivery person based on the ingredient information;

[1669] a delivery staff means for collecting ingredients according to delivery instructions and transporting them to a partner cooking facility;

[1670] A generative AI method to create optimal cooking menus based on ingredient information in the database;

[1671] A means of cooking and storing ingredients at partner cooking facilities,

[1672] The system includes a delivery person means for delivering cooked food based on a user's order.

[1673] (Claim 2)

[1674] The system according to claim 1, further comprising means for determining the freshness of the food ingredient information.

[1675] (Claim 3)

[1676] 2. The system according to claim 1, further comprising a user terminal means for displaying a menu of cooked dishes based on the user's order information.

[1677] "Example 1"

[1678] (Claim 1)

[1679] a user terminal means for photographing surplus ingredients;

[1680] means for transmitting the captured image to a server;

[1681] A server-based AI means for analyzing the transmitted images and registering the food information in a database;

[1682] A server means for sending a collection instruction to a delivery person based on the food ingredient information;

[1683] a delivery staff means for collecting ingredients according to the collection instructions and transporting them to the cooking facility;

[1684] A generative AI method to create optimal cooking menus based on ingredient information in the database;

[1685] A means of preparing and storing food in a cooking facility;

[1686] a delivery person means for delivering cooked food based on an order from a user;

[1687] A means for reporting the status of the food being transported to the cooking facility to the server in real time;

[1688] The system includes a means for updating the server with information regarding completion of storage.

[1689] (Claim 2)

[1690] The system according to claim 1, further comprising means for determining the freshness of the food ingredient information.

[1691] (Claim 3)

[1692] 2. The system according to claim 1, further comprising a user terminal means for displaying a menu of cooked dishes based on the user's order information.

[1693] "Application Example 1"

[1694] (Claim 1)

[1695] a means for taking a photograph of the surplus ingredients using a user terminal and transmitting the image to a server;

[1696] A generative AI model that analyzes the images sent and registers the food information in a database.

[1697] means for generating food delivery instructions based on the ingredient information and transmitting the instructions to a delivery person;

[1698] a means for collecting the ingredients in accordance with the delivery instructions and transporting them to the partner cooking facility;

[1699] A generative AI model that creates optimal cooking menus from ingredient information in a database,

[1700] A means of cooking and storing ingredients at partner cooking facilities,

[1701] means for delivering cooked food based on a user's order;

[1702] A system that includes a means for users to check and order food information and menu items using a smartphone.

[1703] (Claim 2)

[1704] The system according to claim 1, further comprising means for determining the freshness of the food ingredient information.

[1705] (Claim 3)

[1706] 10. The system of claim 1, further comprising a smartphone application means for allowing a user to view and order a menu of prepared dishes based on the order information.

[1707] "Example 2: Combining Emotion Engines"

[1708] (Claim 1)

[1709] a user terminal for taking a photo of surplus ingredients;

[1710] means for transmitting the captured image to a server;

[1711] A server-based AI means for analyzing the transmitted images and registering the food information in a database;

[1712] A server means for sending a collection instruction to a deliverer based on the food ingredient information;

[1713] a delivery means for collecting the ingredients according to the collection instructions and transporting them to a partner cooking facility;

[1714] A generative AI method to create optimal cooking menus based on ingredient information in the database;

[1715] A means of cooking and storing ingredients at partner cooking facilities,

[1716] a delivery means for delivering cooked food based on a user's order;

[1717] emotion analysis means of the user terminal for analyzing the emotional state of the user;

[1718] The system includes a server means for proposing optimal cooking menus based on the analyzed emotional information.

[1719] (Claim 2)

[1720] The system according to claim 1, further comprising means for determining the freshness of the food ingredient information.

[1721] (Claim 3)

[1722] 2. The system according to claim 1, further comprising a user terminal means for displaying a menu of cooked dishes based on the user's order information.

[1723] "Application example 2 when combining emotion engines"

[1724] (Claim 1)

[1725] a user terminal for taking a photo of surplus ingredients;

[1726] means for transmitting the captured image to a server;

[1727] A server-based AI means for analyzing the transmitted images and registering the food information in a database;

[1728] a server means for transmitting a collection instruction to a food delivery person based on the ingredient information;

[1729] a delivery staff means for collecting ingredients according to delivery instructions and transporting them to a partner cooking facility;

[1730] A generative AI method to create optimal cooking menus based on ingredient information in the database;

[1731] A means of cooking and storing ingredients at partner cooking facilities,

[1732] emotion engine means for analyzing an emotional state of a user using a user terminal;

[1733] a server means for suggesting a food menu based on the emotional state of the user;

[1734] The system includes a delivery person means for delivering cooked food based on a user's order.

[1735] (Claim 2)

[1736] The system according to claim 1, further comprising means for determining the freshness of the food ingredient information.

[1737] (Claim 3)

[1738] 2. The system according to claim 1, further comprising a user terminal means for displaying a menu of cooked dishes based on the user's order information. [Explanation of symbols]

[1739] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a user terminal for taking a photo of surplus ingredients; means for transmitting the captured image to a server; A server-based AI means for analyzing the transmitted images and registering the food information in a database; a server means for transmitting a collection instruction to a food delivery person based on the ingredient information; a delivery staff means for collecting ingredients according to delivery instructions and transporting them to a partner cooking facility; A generative AI method to create optimal cooking menus based on ingredient information in the database; A means to prepare and store ingredients in partner cooking facilities, The system includes a delivery person means for delivering cooked food based on a user's order.

2. The system according to claim 1, further comprising means for determining the freshness of the food ingredient information.

3. 2. The system according to claim 1, further comprising a user terminal means for displaying a menu of cooked dishes based on the user's order information.

Images