system

Abstract

We provide the system. [Solution] An input method for entering the user's personal information, A menu generation means that generates a suitable menu based on the aforementioned personal information, A means of presenting the generated menu to the user, Preparation methods for preparing meal kits that correspond to the selected special diet, A delivery means for delivering the meal kit to the user, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] An object of the present invention is to provide an optimal menu based on personal information such as a user's family composition, hobbies, and health condition, and further to comprehensively perform meal kit delivery to reduce the burden of daily meal preparation. In particular, it is an issue to address various needs that vary from household to household, such as preparation of special meals for infants, the elderly requiring care, and diabetics, efficient use of ingredients in the refrigerator, and procurement of appropriate amounts of ingredients for one person or two people.

Means for Solving the Problems

[0005] The present invention solves the above problems by the following means: providing a system including input means for inputting the user's personal information, menu generation means for generating a suitable menu based on the personal information, presentation means for presenting the generated menu to the user, preparation means for preparing a meal kit corresponding to the selected special diet, and delivery means for delivering the meal kit to the user. Furthermore, it includes image analysis means for analyzing images of ingredients in the refrigerator uploaded by the user, auxiliary menu generation means for generating a menu based on the ingredients recognized by the image analysis means, and auxiliary presentation means for presenting the menu generated by the auxiliary menu generation means to the user. In addition, it also includes evaluation data utilization means for further optimizing the menu based on the user's past menu evaluation data. This makes it possible to provide optimal menu suggestions and meal kit delivery services for each individual user, and to reduce the burden of daily meal preparation.

[0006] A "user" is an individual or household that uses this system, enters personal information, and receives menu suggestions and meal kit delivery services.

[0007] "Personal information" refers to personal information such as the user's family structure, age group, preferences, allergy information, and special dietary needs.

[0008] "Input method" refers to the interface that allows users to input personal information into the system.

[0009] "Menu generation means" refers to an algorithm or AI system for generating the optimal menu based on the user's personal information.

[0010] "Presentation means" refers to displays, application screens, etc., used to show the generated menu to the user.

[0011] "Special diets" refer to meals tailored to specific health conditions or needs, such as those for infants, those requiring care, or those with diabetes.

[0012] "Preparation methods" refer to the processes and systems for assembling and preparing meal kits based on selected and confirmed menus.

[0013] "Delivery method" refers to a logistics system or service for delivering prepared meal kits to a location specified by the user.

[0014] "Image analysis means" refers to the technology and algorithms used to analyze images of food items in a refrigerator uploaded by a user and to recognize those items.

[0015] "Auxiliary menu generation means" refers to an algorithm or system for generating menus based on ingredients recognized by image analysis means.

[0016] "Auxiliary presentation means" refers to an interface for displaying menus generated by the auxiliary menu generation means to the user.

[0017] "Methods for using evaluation data" refers to algorithms and systems that utilize users' past menu evaluation data to further optimize menus. [Brief explanation of the drawing]

[0018] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of the data processing device and smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0019] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0020] First, the language used in the following description will be explained.

[0021] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0022] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0023] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0024] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0026] [First Embodiment]

[0027] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0028] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0031] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0034] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0038] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0039] This invention is a system that provides an optimal menu based on the user's personal information, such as family structure, preferences, and health status, and even delivers the meal kit. Specific embodiments of this system are described below.

[0040] First, the user creates an account. The user uses their device to enter their email address and password to create the account. Afterward, the user receives a confirmation email and clicks the link to authenticate their account. Next, they enter personal information such as family structure, age group, preferences, allergy information, and special dietary needs. This information is sent from the user's device to the server and stored in the database.

[0041] Next, the server uses AI to generate the optimal menu based on the stored personal information. For example, if the family consists of two adults and one infant, and their preference is for Japanese food, a menu such as "Teriyaki Chicken," "Spinach with Sesame Dressing," and "Rice" will be generated. The server also takes into account the user's past menu evaluation data to present an even more individually optimized menu.

[0042] The generated menu is presented to the user via a terminal. The user can select a special meal (e.g., a menu for infants) from the presented menu. When the user selects a special meal, that information is sent to the server, which generates a meal kit corresponding to the special meal. For example, for a family with an infant, a meal kit is prepared that includes teriyaki chicken seasoned for infants and soft spinach.

[0043] Next, the user can choose a delivery method, such as "home delivery" or "refrigerated locker at the nearest supermarket." If the user selects "home delivery," that information is sent to the server, and instructions are sent to the logistics system. As a result, the meal kit is delivered to the user's home.

[0044] Furthermore, users can take photos of the ingredients in their refrigerator and upload the images to the system. The device sends the images to the server, which then performs image analysis. For example, if the refrigerator contains cabbage, carrots, and pork, the server recognizes these ingredients and generates menus such as "stir-fried pork and cabbage" and "carrot salad." The generated menus are then presented to the user via the device.

[0045] As a concrete example, the following process can be considered: When the user enters their family composition (2 adults, 1 infant), preferences (likes Japanese food), and allergies (peanuts), the system suggests "Teriyaki Chicken," "Sesame Spinach," and "Rice" based on that information. Furthermore, if the user selects the "Infant Menu" and specifies "Home Delivery," the meal kit will be delivered to the user's home.

[0046] In this way, the system of the present invention can provide personalized menu suggestions and meal kit delivery services tailored to the user's needs, thereby reducing the burden of daily meal preparation.

[0047] The following describes the processing flow.

[0048] Step 1:

[0049] User: Go to the account creation screen and enter your email address and password.

[0050] Terminal: Sends the entered information to the server.

[0051] Server: Registers the email address and password in the authentication system and sends a confirmation email to the user.

[0052] Step 2:

[0053] User: Receive a confirmation email and click the link to verify your account.

[0054] Terminal: Displays an authentication screen and presents a form for entering family composition, age range, preferences, allergy information, and special dietary needs.

[0055] User: Enter information such as family composition (2 adults, 1 infant), age range (30s), preferences (likes Japanese food), and allergies (peanuts) into the input form.

[0056] Terminal: Sends the entered information to the server.

[0057] Server: Stores the received personal information in the database.

[0058] Step 3:

[0059] Server: Retrieves user personal information from the database.

[0060] Server: An AI algorithm generates the optimal menu based on personal information (e.g., "Teriyaki Chicken," "Sesame-dressed Spinach," "Rice").

[0061] Server: It also takes into account the user's past menu evaluation data to generate individually optimized menus (e.g., "Miso Soup").

[0062] Terminal: Displays the suggested menu on the user's screen.

[0063] Step 4:

[0064] User: Select a special meal (e.g., "Infant Menu") from the suggested menus.

[0065] Terminal: Sends the selected option information to the server.

[0066] Server: Generates meal kits tailored to special dietary needs (e.g., "Teriyaki Chicken (seasoned for infants)", "Soft Spinach").

[0067] Step 5:

[0068] User: Select a delivery method and specify either "Home Delivery" or "Refrigerated Locker at the nearest supermarket."

[0069] Terminal: Sends the selected shipping method information to the server.

[0070] Server: Issues instructions to the logistics system and prepares for the meal kits to be delivered to the user's specified location.

[0071] Step 6:

[0072] User: Take a picture of the ingredients in your refrigerator (e.g., "cabbage, carrots, pork") and upload the image to the system.

[0073] Terminal: Sends images to the server.

[0074] Server: Performs image analysis and generates menus based on recognized ingredients (e.g., "Stir-fried pork and cabbage", "Carrot salad").

[0075] Terminal: Displays the newly generated menu on the user's screen.

[0076] (Example 1)

[0077] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0078] Conventional meal planning systems struggle to propose individually optimized menus based on users' personal information, and their suggestions tend to be biased. Furthermore, they often lack effective ways to utilize the ingredients users already possess, leading to food waste. To address these issues, there is a need for a system that generates optimal meal plans tailored to each user's individual circumstances and delivers meal kits based on those plans.

[0079] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0080] In this invention, the server includes input means for inputting the user's personal information, meal plan generation means for generating an optimal meal plan based on the personal information, presentation means for presenting the generated meal plan to the user, preparation means for preparing an ingredient kit corresponding to the selected specific food, and delivery means for delivering the ingredient kit to the user. This enables the generation of a personalized meal plan tailored to the user's individual circumstances and the delivery of an ingredient kit based on that meal plan.

[0081] "User personal information" refers to information such as the user's family structure, age group, preferences, allergy information, and special dietary needs.

[0082] "Input method" refers to an interface for users to enter personal information, including input forms and applications on a device.

[0083] "Meal plan generation method" refers to algorithms or AI models that generate optimal meal plans based on the user's personal information.

[0084] "Presentation means" refers to a method for displaying the generated meal plan to the user, and includes the terminal's display and notification system.

[0085] "Specific foods" refer to ingredients or menu items selected by users based on their special needs or requests.

[0086] "Preparation means" refers to a system for preparing ingredient kits corresponding to selected specific foods, including the selection and packing of ingredients.

[0087] "Delivery method" refers to the means of delivering prepared food kits to a location specified by the user, and includes logistics systems and delivery services.

[0088] "Images of food in storage" refers to images taken by users of food in their home refrigerators or food storage areas.

[0089] "Image analysis means" refers to a technology for analyzing food images uploaded by users and recognizing the ingredients contained within them, and includes an image recognition algorithm.

[0090] "Supplementary meal plan generation means" refers to algorithms or AI models for generating supplementary meal plans based on ingredients recognized by image analysis means.

[0091] "Auxiliary presentation means" refers to a method for displaying a meal plan generated by the auxiliary meal plan generation means to the user, and includes the terminal's display and notification system.

[0092] "Methods for using evaluation data" refers to systems and algorithms that utilize evaluation data from users' past meal plans to further optimize meal plans.

[0093] This invention is a system that provides an optimal meal plan based on the user's personal information and even delivers the ingredient kit. This system consists of three components: a server, a terminal, and the user. Specific embodiments of this system are described below.

[0094] First, the user creates an account using their device. The user enters their email address and password and registers on the account creation page. This information is sent from the device to the server, which stores it in its database. The server then sends a confirmation email to the user, and the user clicks the confirmation link in the email to authenticate their account.

[0095] Next, the user logs into their account and enters personal information such as family structure, age range, preferences, and allergy information. The entered information is sent from the device to the server and stored in the database.

[0096] The server uses an AI model to generate an optimal meal plan based on stored personal information. For example, if the family consists of two adults and one infant, prefers Japanese food, and has a peanut allergy, a menu such as "Teriyaki Chicken," "Spinach with Sesame Dressing," and "Rice" will be suggested. Because this process uses a generative AI model, the following input is used as a prompt.

[0097] Example of a prompt:

[0098] "The user's family consists of two adults and one infant, and they prefer Japanese food. They also have a peanut allergy. Please suggest the best menu based on these conditions."

[0099] The generated meal plan is sent from the server to the user's terminal, where the user can view the suggested menu on their terminal screen. The user can select specific foods from this menu (for example, infant menus), and this selection information is sent from the terminal to the server. Based on this information, the server prepares ingredient kits corresponding to the selected foods.

[0100] Furthermore, users can choose their delivery method. For example, they might have options like "home delivery" or "refrigerated locker at the nearest supermarket." The selected delivery method is then sent from the terminal to the server. The server sends instructions to the logistics system, and the meal kit is delivered to the user using the specified delivery method.

[0101] To utilize the ingredients in their storage, users take photos of the food in their refrigerator and upload the images to their device. The device sends these images to a server, which performs image analysis to recognize the ingredients. For example, if the refrigerator contains cabbage, carrots, and pork, the server will suggest supplementary menus based on these ingredients, such as "stir-fried pork and cabbage" or "carrot salad." This information is also sent from the server to the user's device.

[0102] This system supports users in their daily meal preparation and contributes to reducing food waste by generating an optimal meal plan tailored to each user's individual needs and providing delivery of ingredient kits based on that plan.

[0103] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0104] Step 1:

[0105] The user enters their email address and password on the device's registration screen and presses the "Create Account" button. This information is sent from the device to the server. The input includes an email address (e.g., example@example.com) and a password. An account creation request is sent to the server as output.

[0106] Step 2:

[0107] The server stores the received account information in a database and sends a confirmation email to the user's email address. As part of data processing, the email address and password are hashed and stored. The confirmation email is then sent to the user as output.

[0108] Example: The server stores the email address in a database and sends a confirmation email using the SMTP protocol.

[0109] Step 3:

[0110] The user clicks the link in the confirmation email to authenticate their account. This action sends the user's request back to the server, updating the account status to "Authenticated." The input is the click of the email link, and the output is the updated account authentication status.

[0111] Step 4:

[0112] After logging in, users enter personal information such as family structure, age group, preferences, and allergy information. This information is sent from the device to the server and stored in the database. Personal information is included as input, and the user profile is updated as output.

[0113] Example: The user enters information such as "2 adults, 1 infant," "likes Japanese food," and "peanut allergy."

[0114] Step 5:

[0115] The server uses an AI model to generate an optimal meal plan based on stored personal information. As part of the data processing, personal information is converted into prompt sentences and input into the AI ​​model. The generated meal plan is obtained as output.

[0116] Example prompt: "The user's family consists of two adults and one infant, and they prefer Japanese food. They have a peanut allergy. Please suggest the best menu based on these conditions."

[0117] Step 6:

[0118] The server sends the generated meal plan to the user's terminal. As output, the meal plan data is sent to the terminal.

[0119] Example: The server sends data in JSON format and converts it for display on the terminal.

[0120] Step 7:

[0121] The user reviews the suggested meal plan via their device and selects specific foods (e.g., infant menu). This selection information is sent from the device to the server as input, and then saved to the server as output.

[0122] Step 8:

[0123] The server generates information to prepare an ingredient kit corresponding to the selected specific food item. The kit contents are determined based on the specific food item information as part of the data processing. Preparation instructions are then generated as output.

[0124] Example: Modify the seasoning of "Teriyaki Chicken" to suit infants and prepare a meal kit that includes soft spinach.

[0125] Step 9:

[0126] The user selects a shipping method and sends the information from their device to the server. The input includes the selected shipping method, and the output is stored on the server.

[0127] Step 10:

[0128] The server issues instructions to the logistics system, which then delivers the food kits to the user using the specified delivery method. The data calculation generates delivery instructions for the logistics system, and the output is the execution of those instructions.

[0129] Example: A meal kit is delivered to the user's home.

[0130] Step 11:

[0131] The user takes a picture of the food in the refrigerator and uploads the image to their device. The input includes the captured image data, and the output is the image data sent to the server.

[0132] Step 12:

[0133] The server performs image analysis to recognize ingredients from the input image. As part of data processing, it analyzes the image data and extracts ingredient information. The output is a list of the recognized ingredients.

[0134] Example: Image analysis identifies cabbage, carrots, and pork.

[0135] Step 13:

[0136] The server generates a supplementary meal plan based on the recognized ingredients and sends it to the user's terminal. As part of data processing, the recognized ingredient information is input into an AI model to generate a supplementary meal plan. As output, the supplementary meal plan is generated and sent to the terminal.

[0137] Examples: "Stir-fried pork and cabbage" and "Carrot salad" are suggested.

[0138] In this way, by adding specific actions at each processing step and clearly indicating data input and output, personalized meal planning and ingredient kit delivery tailored to the user's needs can be realized.

[0139] (Application Example 1)

[0140] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0141] While conventional systems offered menu suggestions and meal kit delivery based on users' personal information, they lacked the functionality to effectively utilize ingredients in the refrigerator and propose more precisely personalized menus. Furthermore, the lack of integrated food delivery services that allowed users to directly order meals based on the suggested menus meant that user convenience was not sufficiently improved. Therefore, there is a need for a system that enables food suggestions and delivery based on individual user needs and real-time circumstances.

[0142] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0143] In this invention, the server includes input means for inputting the user's personal information, menu generation means for generating a suitable menu based on the personal information, presentation means for presenting the generated menu to the user, preparation means for preparing a meal kit corresponding to the selected special diet, delivery means for delivering the meal kit to the user, image analysis means for uploading images of ingredients in the refrigerator and analyzing those images, auxiliary menu generation means for generating auxiliary menus based on the ingredients recognized by the image analysis means, auxiliary presentation means for presenting the auxiliary menus generated by the auxiliary menu generation means to the user, and ordering means for ordering dishes based on the provided menu. As a result, the user can have an optimal menu suggested based on their personal information and the ingredients in their refrigerator, and can also directly order dishes according to that menu.

[0144] "User personal information" refers to individual information such as the user's family structure, age group, preferences, allergy information, and special dietary needs.

[0145] An "input method" is an interface for users to input personal information, and may consist of forms provided on the screen of a smartphone or computer, for example.

[0146] A "menu generation method" is a system that uses artificial intelligence (AI) to generate the optimal menu based on the user's personal information.

[0147] "Presentation means" refers to an interface for displaying the generated menu to the user, and includes the display of a smartphone or tablet.

[0148] "Preparation methods" refer to the means of preparing meal kits that correspond to the special diet selected by the user.

[0149] "Delivery method" refers to the means by which meal kits are delivered to users, including logistics systems and delivery companies.

[0150] "Image analysis means" refers to a technology for analyzing images of food items inside a refrigerator and recognizing them, and is a system that utilizes computer vision and machine learning algorithms.

[0151] The "supplementary menu generation means" is a means for generating additional menus based on ingredients recognized by the image analysis means.

[0152] The "auxiliary presentation means" is an interface for presenting menus generated by the auxiliary menu generation means to the user.

[0153] "Ordering method" refers to a means of ordering food based on the provided menu, and includes, for example, the ordering screen of a smartphone application.

[0154] This invention is a system that provides optimal menus based on the user's personal information and also delivers meal kits and food. Specific embodiments of this system are described below.

[0155] First, users create an account using a smartphone or computer application, entering their email address and password. Once the account is created, a confirmation email is sent. Users then click the link in the email to verify their account.

[0156] Next, the user enters personal information such as family structure, age group, preferences, allergy information, and special dietary needs. This information is sent from the user's device to the server and stored in the database.

[0157] The server uses AI (artificial intelligence) to generate the optimal menu based on stored personal information. For example, if the family consists of two adults and one infant, their preference is for Japanese food, and they have peanut allergies, the server will generate a menu such as "Teriyaki Chicken," "Spinach with Sesame Dressing," and "Rice."

[0158] The generated menu is presented via the user's terminal. The user can select a special meal option (e.g., a menu for infants) from the presented menu. When the user selects a special meal, that information is sent to the server, which then prepares a meal kit corresponding to that special meal. For example, a meal kit is created that includes a menu with modified seasoning for infants.

[0159] Furthermore, users take photos of the ingredients in their refrigerator and upload the images to the application. The user's device sends the images to a server, which performs image analysis. For example, if the refrigerator contains cabbage, carrots, and pork, the server recognizes these ingredients and generates suggested menus such as "stir-fried pork and cabbage" and "carrot salad."

[0160] This ensures that fresh, personalized menus are always generated and presented to the user. These supplementary menus are presented through the user's device.

[0161] Users can order food directly based on the presented menu. The order information is sent to the server, and the logistics system delivers the food to the user's specified location (home or a refrigerated locker at the nearest supermarket).

[0162] To implement this system, software libraries such as "Python," "Requests," and "PIL" are used, along with devices such as smartphones and tablets.

[0163] As a concrete example, suppose a user inputs, "Please suggest a menu using the pork, cabbage, and carrots I have in my refrigerator." In response, the AI ​​uses the following prompt:

[0164] Prompt message:

[0165] I have pork, cabbage, and carrots in the refrigerator. My family consists of two adults and one infant. They have a peanut allergy. Please suggest a menu they can enjoy.

[0166] In this way, the system of the present invention provides users with personalized menu suggestions and meal kit delivery services, thereby reducing the burden of daily meal preparation.

[0167] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0168] Step 1:

[0169] Users create an account using a smartphone or computer application, entering their email address and password. The entered information is sent from the device to the server. The server stores the user's account information in a database and generates and sends a confirmation email to the user. The user authenticates their account by clicking a link in the confirmation email. In this step, the input is the email address and password, and the output is the authenticated user account.

[0170] Step 2:

[0171] Users enter personal information into the application, such as family structure, age group, preferences, allergy information, and special dietary needs. This information is sent from the terminal to the server and stored in a database. In this step, the input is personal information, and the output is the stored personal information.

[0172] Step 3:

[0173] The server uses AI (artificial intelligence) to generate the optimal menu based on stored personal information. The AI ​​generates prompt messages that take into account the user's family structure and preferences. For example, it might generate a prompt message such as, "Please suggest the optimal menu for a user with a family structure of 2 adults and 1 infant, a preference for Japanese food, and peanut allergies," and input this into the AI ​​model. The output is an optimized menu. In this step, the input is personal information, and the output is an optimized menu.

[0174] Step 4:

[0175] The server sends the generated optimized menu to the user's terminal. The terminal displays the suggested menu to the user. The user can select special meals, etc., from the presented menu. In this step, the input is the optimized menu, and the output is the menu displayed on the user's terminal.

[0176] Step 5:

[0177] The user takes a picture of the food in their refrigerator with their camera and uploads the image to the application. The image is sent from the device to the server. The server uses image analysis technology to recognize the food in the image. In this step, the input is an image of the food in the refrigerator, and the output is data of the recognized food.

[0178] Step 6:

[0179] The server generates a suggested menu based on the recognized ingredient data. For example, it generates a prompt such as, "There is pork, cabbage, and carrots in the refrigerator. Please suggest a menu using these ingredients," and inputs it into the AI ​​model. The output is a suggested menu. In this step, the input is the recognized ingredient data, and the output is a suggested menu.

[0180] Step 7:

[0181] The server sends the generated supplementary menu to the user's terminal. The terminal displays the suggested supplementary menu to the user. In this step, the input is the supplementary menu, and the output is the supplementary menu displayed on the user's terminal.

[0182] Step 8:

[0183] Users can order dishes based on the presented menu and supplementary menus. Order information is sent from the terminal to the server, which prepares the dishes via a logistics system and delivers them to the user's specified location. In this step, the input is the order information, and the output is the dishes delivered to the user's specified location.

[0184] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0185] This invention is a system that provides an optimal menu considering not only the user's family structure, preferences, and health condition, but also the user's emotions, and even delivers the meal kit. Specific embodiments of this system are described below.

[0186] First, the user creates an account. The user uses their device to enter their email address and password to create the account. Afterward, the user receives a confirmation email and clicks the link to authenticate their account. Next, they enter personal information such as family structure, age group, preferences, allergy information, and special dietary needs. This information is sent from the user's device to the server and stored in the database.

[0187] Next, the server uses AI to generate the optimal menu based on the stored personal information. For example, if the family consists of two adults and one infant, and their preference is for Japanese food, a menu such as "Teriyaki Chicken," "Spinach with Sesame Dressing," and "Rice" will be generated. The server also takes into account the user's past menu evaluation data to present an even more individually optimized menu.

[0188] The generated menu is presented to the user via a terminal. The user can select a special meal (e.g., a menu for infants) from the presented menu. When the user selects a special meal, that information is sent to the server, which generates a meal kit corresponding to the special meal. For example, for a family with an infant, a meal kit is prepared that includes teriyaki chicken seasoned for infants and soft spinach.

[0189] Next, the user can choose a delivery method, such as "home delivery" or "refrigerated locker at the nearest supermarket." If the user selects "home delivery," that information is sent to the server, and instructions are sent to the logistics system. As a result, the meal kit is delivered to the user's home.

[0190] Furthermore, users can take photos of the ingredients in their refrigerator and upload the images to the system. The device sends the images to the server, which then performs image analysis. For example, if the refrigerator contains cabbage, carrots, and pork, the server recognizes these ingredients and generates menus such as "stir-fried pork and cabbage" and "carrot salad." The generated menus are then presented to the user via the device.

[0191] Furthermore, this system includes an emotion engine that recognizes the user's emotions. Users can communicate their current mood and feelings to the system using an emotion input interface. For example, if a user inputs an emotion such as "I'm feeling stressed," the server will adjust the menu based on that information. For instance, it can suggest menus using ingredients with relaxing effects or recommend the user's favorite comfort foods.

[0192] The emotion engine continuously collects user emotional data and uses it for long-term preference analysis. This allows it to learn what emotions users have felt and which menu items they have preferred in the past, enabling it to provide even more personalized suggestions.

[0193] For example, if a user enters their family structure (2 adults, 1 infant), preferences (likes Japanese food), and allergies (peanuts), the system will suggest "Teriyaki Chicken," "Sesame Spinach," and "Rice." Furthermore, if the user selects the "Infant Menu" and specifies "Home Delivery," the meal kit will be delivered to the user's home. Also, if the user enters that they are "feeling stressed," the system will suggest additional items such as a "Matcha-flavored dessert" that has a relaxing effect.

[0194] In this way, the system of the present invention provides personalized menu suggestions and meal kit delivery services tailored to the user's needs and emotions, thereby reducing the burden of daily meal preparation and improving the quality of life.

[0195] The following describes the processing flow.

[0196] Step 1:

[0197] User: Go to the account creation screen and enter your email address and password.

[0198] Terminal: Sends the entered information to the server.

[0199] Server: Registers the email address and password in the authentication system and sends a confirmation email to the user.

[0200] Step 2:

[0201] User: Receive a confirmation email and click the link to verify your account.

[0202] Terminal: Displays an authentication screen and presents a form for entering family composition, age range, preferences, allergy information, and special dietary needs.

[0203] User: Enter information such as family composition (2 adults, 1 infant), age range (30s), preferences (likes Japanese food), and allergies (peanuts) into the input form.

[0204] Terminal: Sends the entered information to the server.

[0205] Server: Stores the received personal information in the database.

[0206] Step 3:

[0207] Server: Retrieves user personal information from the database.

[0208] Server: An AI algorithm generates the optimal menu based on personal information (e.g., "Teriyaki Chicken," "Sesame-dressed Spinach," "Rice").

[0209] Server: It also takes into account the user's past menu evaluation data to generate individually optimized menus (e.g., "Miso Soup").

[0210] Terminal: Displays the suggested menu on the user's screen.

[0211] Step 4:

[0212] User: Select a special meal (e.g., "Infant Menu") from the suggested menus.

[0213] Terminal: Sends the selected option information to the server.

[0214] Server: Generates meal kits tailored to special dietary needs (e.g., "Teriyaki Chicken (seasoned for infants)", "Soft Spinach").

[0215] Step 5:

[0216] User: Select a delivery method and specify either "Home Delivery" or "Refrigerated Locker at the nearest supermarket."

[0217] Terminal: Sends the selected shipping method information to the server.

[0218] Server: Issues instructions to the logistics system and prepares for the meal kits to be delivered to the user's specified location.

[0219] Step 6:

[0220] User: Take a picture of the ingredients in your refrigerator (e.g., "cabbage, carrots, pork") and upload the image to the system.

[0221] Terminal: Sends images to the server.

[0222] Server: Performs image analysis and generates menus based on recognized ingredients (e.g., "Stir-fried pork and cabbage", "Carrot salad").

[0223] Terminal: Displays the newly generated menu on the user's screen.

[0224] Step 7:

[0225] User: Use the emotion input interface to enter your current mood and emotions into the system (e.g., "I'm feeling stressed").

[0226] Terminal: Sends emotional data to the server.

[0227] Server: The emotion engine analyzes emotional data and adjusts the menu based on emotions (e.g., a relaxing "matcha dessert").

[0228] Terminal: Displays the adjusted new menu on the user's screen.

[0229] (Example 2)

[0230] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0231] Conventional menu generation systems suggest menus based on users' personal information and preferences, but they are insufficient in generating personalized menus that take into account users' emotions and refrigerator inventory, and in properly managing and delivering them. Furthermore, they lack sufficient functionality to optimize menus over the long term by utilizing users' past menu evaluations and emotional data, making it difficult to continuously increase user satisfaction.

[0232] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0233] In this invention, the server includes input means for inputting the user's personal information, menu generation means for generating a suitable menu based on the personal information and the user's emotional data, presentation means for presenting the generated menu to the user, preparation means for preparing a meal kit corresponding to a special meal selected by the user, delivery means for delivering the meal kit to the user, image analysis means for analyzing images of ingredients in the refrigerator uploaded by the user, auxiliary menu generation means for generating a menu based on the ingredients recognized by the image analysis means, auxiliary presentation means for presenting the generated auxiliary menu to the user, and evaluation data utilization means for further optimizing the menu based on the user's past menu evaluation data and emotional data. This makes it possible to provide and deliver optimized and personalized menus, and meal kits based on them, taking into account the user's personal information, emotional data, and the inventory in the refrigerator.

[0234] - A "user" refers to an individual who uses this system to create an account and provides personal information and sentiment data.

[0235] "Personal information" refers to information about the user, such as family structure, age group, preferences, allergy information, and special dietary needs.

[0236] "Emotional data" refers to information that users input, indicating their current mood and emotions.

[0237] "Menu generation means" refers to a function that generates appropriate menus based on personal information and emotional data.

[0238] "Presentation means" refers to a function for notifying or displaying the generated menu to the user.

[0239] "Special meals" refer to meal menus tailored to specific needs selected by the user.

[0240] "Preparation means" refers to the function of preparing meal kits that accommodate special dietary needs.

[0241] "Delivery method" refers to the function for delivering prepared meal kits to users.

[0242] "Image analysis means" refers to a function that analyzes images of food items in a refrigerator uploaded by the user to recognize the food items.

[0243] "Auxiliary menu generation means" refers to a function for generating menus based on ingredients recognized by image analysis means.

[0244] "Auxiliary presentation means" refers to a function for notifying or displaying the generated auxiliary menu to the user.

[0245] "Means of using evaluation data" refers to functions that further optimize menus based on the user's past menu evaluation data and sentiment data.

[0246] A "meal kit" refers to a set of ingredients prepared based on a pre-designed menu.

[0247] "Food in the refrigerator" refers to the food stored in the user's refrigerator.

[0248] This invention is a system that provides an optimal menu considering the user's personal information and emotional data, and even delivers the meal kit. Specifically, the system operates in the following steps.

[0249] First, the user accesses the system's webpage or app using their device. The user enters their email address and password to create an account. The server receives the entered information, registers the new user information in the database, and then sends a confirmation email. The user clicks the link in the email to authenticate their account. This completes the account creation process.

[0250] Next, the user enters personal information from their device, such as family structure, preferences, allergy information, and special diet needs. This information is sent to the server and stored in the database. For example, if the family consists of two adults and one infant, preferences are for Japanese food, and allergies include peanuts, this data will be stored in the user's profile.

[0251] The server uses stored personal information to call up an AI model (such as OpenAI's GPT-4) to generate an optimal menu. For example, if a family consists of two adults and one infant and prefers Japanese food, the AI ​​will suggest a menu such as "teriyaki chicken," "spinach with sesame dressing," and "rice." The server saves the generated menu in a database and presents it to the user via the terminal.

[0252] Users can review the menu and select special meals (e.g., infant menus). The selected information is sent to the server, which then prepares a meal kit tailored to the special meal. For example, the server might prepare a meal kit of "Teriyaki Chicken" with the seasoning modified for infants and include "Soft Spinach."

[0253] Next, the user selects a delivery method using a terminal. Options include "home delivery" and "refrigerated locker at the nearest supermarket." If the user selects "home delivery," that information is sent to the server. The server then instructs the logistics system to deliver the meal kit to the location specified by the user.

[0254] Furthermore, the user takes pictures of the food in the refrigerator with their device's camera and uploads the images to the server. The server uses an image analysis engine (such as the Google Cloud Vision API) to recognize the food. Based on the recognized food, the server uses an auxiliary menu generation method to generate the optimal menu and presents it to the user. For example, if the refrigerator contains cabbage, carrots, and pork, menus such as "stir-fried pork and cabbage" and "carrot salad" will be suggested.

[0255] Users input their current mood and emotions through their device. For example, they might input "I'm feeling stressed" or "I'm tired." The server adjusts the menu based on the emotional input data. For instance, if the user is feeling stressed, a relaxing dessert such as a "matcha-flavored dessert" might be suggested.

[0256] The server continuously collects user emotional data and provides even more personalized suggestions based on long-term preference analysis. For example, if a user previously preferred a particular menu item when they were feeling stressed, this information is used to optimize future suggestions.

[0257] Specific example:

[0258] The user enters their family composition (2 adults, 1 infant), preferences (Japanese food), and allergies (peanuts) into the terminal. The server receives this information, saves it to a database, and then calls an AI model to generate menus such as "Teriyaki Chicken," "Spinach with Sesame Dressing," and "Rice." The user reviews the suggestions on the terminal and selects a special meal (for the infant). Based on this information, the server prepares a meal kit, and if the user selects "Home Delivery," the meal kit is delivered. Additionally, if the user enters "I'm feeling stressed," the server will suggest additional menu items such as a relaxing "Matcha-flavored dessert."

[0259] Example of a prompt:

[0260] The user has entered their family structure (2 adults, 1 infant) and preferred Japanese food. Peanut allergy information has also been provided. Please suggest the most suitable menu and delivery options.

[0261] Thus, the system of the present invention provides personalized menu suggestions and meal kit delivery services tailored to the user's needs and emotions, thereby reducing the burden of daily meal preparation and improving the quality of life.

[0262] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0263] Step 1:

[0264] User account creation

[0265] The user accesses the system's webpage or app using their device. The user enters their email address and password and submits the account creation form. The server receives this information and registers the new user information in its database. The server sends a confirmation email to the registered email address. The user clicks the link in the email to authenticate their account, completing the account creation process.

[0266] Input: User's email address, password

[0267] Output: Account creation confirmation email

[0268] Step 2:

[0269] Input and storage of user personal information

[0270] Users use their devices to input personal information such as family structure, preferences, allergy information, and special dietary needs. The entered information is sent to the server, which stores it in a database. For example, if a user inputs that their family consists of two adults and one infant, their preferences are for Japanese food, and their allergy is peanuts, that information will be saved in the user's profile.

[0271] Input: Family composition, preferences, allergy information, special diet needs

[0272] Output: Personal information stored in the database

[0273] Step 3:

[0274] AI-powered optimal menu generation

[0275] Based on the stored personal information, the server calls the AI model to be used (e.g., generative AI model). The AI generates an optimal menu based on the personal information and sentiment data. For example, if the family composition is two adults and one infant, and the preference is Japanese cuisine, the AI proposes a menu such as "Teriyaki Chicken", "Sesame Spinach", and "Rice". The server saves the generated menu in the database.

[0276] Input: Personal Information

[0277] Output: Optimal Menu

[0278] Step 4:

[0279] Selection of Menu by User

[0280] The user checks the proposed menu through the terminal. When selecting a special diet (e.g., menu for infants), the selection information is sent to the server, and the server saves it in the database. The process of generating a meal kit based on the special diet information is initiated.

[0281] Input: Proposed Menu, Selection of Special Diet

[0282] Output: Special Diet Information Saved in Server

[0283] Step 5:

[0284] Generation of Meal Kit Based on Special Diet

[0285] The server generates an optimized meal kit based on the special diet information selected by the user. For example, it modifies the seasoning of "Teriyaki Chicken" for infants and prepares a meal kit including "Soft Spinach". These information is managed by the server and used for cooperation with the logistics system.

[0286] Input: Special Diet Information

[0287] Output: Optimized Meal Kit Information

[0288] Step 6:

[0289] User selection of delivery method and delivery of meal kits

[0290] The user selects a delivery method using a terminal. Options include "home delivery" and "refrigerated locker at the nearest supermarket." If the user selects "home delivery," that information is sent to the server. The server then instructs the logistics system, and the meal kit is delivered to the location specified by the user.

[0291] Input: Select shipping method

[0292] Output: Delivery Instructions

[0293] Step 7:

[0294] Image recognition of food items inside a refrigerator

[0295] The user takes a picture of the food in their refrigerator with their camera and uploads the image from their device to the server. The server uses an image analysis engine (e.g., an image analysis API) to recognize the food in the image. Based on the recognized food, the server generates an optimal menu and presents it to the user.

[0296] Input: Image of food items in the refrigerator

[0297] Output: Optimal menu

[0298] Step 8:

[0299] User emotional input and menu adjustment

[0300] Users input their current mood and emotions through their device, such as "I'm stressed" or "I'm tired." The device sends this information to a server. The server adjusts the menu based on the emotional input data. For example, if the user is feeling stressed, a relaxing dessert such as a "matcha-flavored dessert" might be suggested.

[0301] Input: Current mood and emotions

[0302] Output: Adjusted menu

[0303] Step 9:

[0304] Accumulation of emotional data and long-term preference analysis

[0305] The server continuously accumulates the user's emotional data. The stored emotional data is used for long-term preference analysis. This enables learning about what kind of emotions the user had and which menu they preferred in the past, making it possible to provide more personalized suggestions. For example, if the user preferred a specific menu when they felt "stressed" in the past, future suggestions are optimized based on this information.

[0306] ​​​​​​​​​​​​​​​​​​​​​In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes an input means for inputting the user's personal information, a menu generation means for generating a suitable menu based on the personal information, a presentation means for presenting the generated menu to the user, a preparation means for preparing a meal kit corresponding to the selected special diet, a delivery means for delivering the meal kit to the user, an emotion input means for inputting or recognizing the user's emotions, and an emotion adjustment means for adjusting the menu based on the emotion input means. This enables personalized menu suggestions that respond to the user's individual needs and emotions, as well as efficient delivery of meal kits.

[0312] "User personal information" refers to information related to an individual user, such as family structure, age group, preferences, allergy information, and special dietary needs.

[0313] An "input method" refers to an interface or device that allows a user to input personal information or emotional data into a system.

[0314] A "menu generation method" is a means for generating an optimal menu based on the user's personal information and emotional data.

[0315] "Presentation means" refers to means of presenting the generated menu to the user visually or audibly.

[0316] "Preparation methods" refer to the means of preparing meal kits that correspond to the selected special diet.

[0317] "Delivery method" refers to the means by which prepared meal kits are delivered to users.

[0318] An "emotion input method" is a means of inputting or recognizing a user's emotions.

[0319] "Emotion adjustment methods" refer to methods for adjusting menus based on the user's emotional data.

[0320] "Image analysis means" refers to a means for analyzing images of food items inside a refrigerator that have been uploaded by a user.

[0321] The "supplementary menu generation means" is a means for generating supplementary menus based on ingredients recognized by the image analysis means.

[0322] "Auxiliary presentation means" refers to a means for presenting menus generated by the auxiliary menu generation means to the user.

[0323] This invention is a system that generates menus and provides / delivers meal kits, taking into account the user's family structure, preferences, health status, and emotions. The system operates when the user creates an account via smartphone, enters personal information, uploads images of ingredients in their refrigerator, or enters their emotions.

[0324] First, users create an account using their smartphone. They enter their email address and password to create the account. Next, they enter personal information such as family structure, age group, preferences, allergy information, and special dietary needs. This information is sent from the device to the server and stored in the database.

[0325] After personal information is saved, the server uses AI to generate an optimal menu based on the saved information. For example, if the family consists of two adults and one infant, and their preference is for Japanese food, a menu such as "Teriyaki Chicken," "Spinach with Sesame Dressing," and "Rice" will be generated. Furthermore, the menu is adjusted using an emotion input method that allows the user to input or recognize their emotions. If the user inputs that they are feeling stressed, the menu will suggest dishes using ingredients with relaxing effects or their favorite comfort foods.

[0326] The generated menu is presented to the user via a terminal. The user can select a special meal (for example, a menu for infants) from the presented menu. When the user selects a special meal, that information is sent to the server, which generates a meal kit corresponding to the special meal. For example, for a family with an infant, a meal kit is prepared that includes teriyaki chicken seasoned for infants and soft spinach. Next, the user selects a delivery method and chooses "home delivery," at which point the meal kit is delivered to the user's home.

[0327] Furthermore, users can take photos of the ingredients in their refrigerator and upload the images to the system. The device sends the images to the server, which then performs image analysis. For example, if the refrigerator contains cabbage, carrots, and pork, the server recognizes these ingredients and generates menus such as "stir-fried pork and cabbage" and "carrot salad." The generated menus are then presented to the user via the device.

[0328] The hardware and software used include user terminals (e.g., smartphones), image recognition technology (e.g., OpenCV, TensorFlow), and emotion recognition technology (e.g., EmotionRecognition model). Menus are generated using AI, and deep learning models (e.g., TensorFlow or PyTorch) are used on the server side.

[0329] For example, if a user enters their family structure (2 adults, 1 infant), preferences (likes Japanese food), and allergies (peanuts), the system will suggest "Teriyaki Chicken," "Sesame Spinach," and "Rice." Furthermore, if the user selects the "Infant Menu" and specifies "Home Delivery," the meal kit will be delivered to the user's home. Also, if the user enters that they are "feeling stressed," the system will suggest additional items such as a "Matcha-flavored dessert" that has a relaxing effect.

[0330] Examples of specific prompt statements include the following:

[0331] "Based on the following profile and emotional data, please suggest the following menu: User Profile: Family Composition: 2 adults, 1 infant; Preferences: Japanese food; Allergies: Peanuts; Special Diet Needs: Infant menu; Emotions: Feeling stressed."

[0332] By fulfilling these requirements, the system can provide personalized menu suggestions and meal kit delivery services tailored to the user's needs and emotions.

[0333] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0334] Step 1:

[0335] The user creates an account using their smartphone. The input consists of an email address and password, which the device receives and sends to the server. The server receives this information, registers the user account in its database, and sends a confirmation email. The output is a confirmation email indicating successful account creation.

[0336] Step 2:

[0337] The user enters personal information such as family structure, age group, preferences, allergy information, and special dietary needs. This information is sent from the terminal to the server, which stores this data in a database. The output confirms the saving of the personal information.

[0338] Step 3:

[0339] The server uses AI to generate the optimal menu based on stored personal information. In this process, personal information stored in a database is used as input, and a generation AI model is used to calculate and generate the menu. The output is the generated menu data.

[0340] Step 4:

[0341] The terminal displays the generated menu to the user. The input is menu data received from the server, which the terminal displays to the user. The output is the menu presented to the user.

[0342] Step 5:

[0343] The user selects a special meal from the presented menu. The user's selection is sent from the terminal to the server, which verifies this information. The input is the user's selected special meal information, and the server prepares the meal kit based on it. The output is the data for the prepared meal kit.

[0344] Step 6:

[0345] The user selects a shipping method. The user's selected shipping method is sent from the terminal to the server, which then issues instructions to the logistics system. The input is the user's selected shipping method, and the output is a notification that the shipping arrangements have been completed.

[0346] Step 7:

[0347] The user takes a picture of the food in their refrigerator and uploads the image from their device to the server. The server uses image analysis to recognize the food. The input is an image of the food in the refrigerator, and the output is the analyzed food data.

[0348] Step 8:

[0349] The server generates supplementary menus based on recognized ingredient data. In this process, the analyzed ingredient data is used as input, and a generation AI model is used to calculate and generate the supplementary menus. The output is the data of the generated supplementary menus.

[0350] Step 9:

[0351] The terminal displays the generated supplementary menu to the user. The input is the supplementary menu data received from the server, which the terminal displays to the user. The output is the supplementary menu presented to the user.

[0352] Step 10:

[0353] The user inputs an emotion or uploads a facial image. The device sends this data to the server, which uses the emotion input method to recognize the emotion. The input is the user's emotion or facial image data, and the output is the recognized emotion data.

[0354] Step 11:

[0355] The server adjusts the menu based on recognized emotion data. This process uses emotion data as input and readjusts the menu using emotion adjustment mechanisms. The output is the adjusted menu data.

[0356] This processing flow enables the delivery of meal kits and menu suggestions optimized to the individual needs and emotions of each user.

[0357] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0358] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0359] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0360] [Second Embodiment]

[0361] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0362] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0363] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0364] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0365] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0366] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0367] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0368] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0369] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0370] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0371] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0372] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0373] This invention is a system that provides an optimal menu based on the user's personal information, such as family structure, preferences, and health status, and even delivers the meal kit. Specific embodiments of this system are described below.

[0374] First, the user creates an account. The user uses their device to enter their email address and password to create the account. Afterward, the user receives a confirmation email and clicks the link to authenticate their account. Next, they enter personal information such as family structure, age group, preferences, allergy information, and special dietary needs. This information is sent from the user's device to the server and stored in the database.

[0375] Next, the server uses AI to generate the optimal menu based on the stored personal information. For example, if the family consists of two adults and one infant, and their preference is for Japanese food, a menu such as "Teriyaki Chicken," "Spinach with Sesame Dressing," and "Rice" will be generated. The server also takes into account the user's past menu evaluation data to present an even more individually optimized menu.

[0376] The generated menu is presented to the user via a terminal. The user can select a special meal (e.g., a menu for infants) from the presented menu. When the user selects a special meal, that information is sent to the server, which generates a meal kit corresponding to the special meal. For example, for a family with an infant, a meal kit is prepared that includes teriyaki chicken seasoned for infants and soft spinach.

[0377] Next, the user can choose a delivery method, such as "home delivery" or "refrigerated locker at the nearest supermarket." If the user selects "home delivery," that information is sent to the server, and instructions are sent to the logistics system. As a result, the meal kit is delivered to the user's home.

[0378] Furthermore, users can take photos of the ingredients in their refrigerator and upload the images to the system. The device sends the images to the server, which then performs image analysis. For example, if the refrigerator contains cabbage, carrots, and pork, the server recognizes these ingredients and generates menus such as "stir-fried pork and cabbage" and "carrot salad." The generated menus are then presented to the user via the device.

[0379] As a concrete example, the following process can be considered: When the user enters their family composition (2 adults, 1 infant), preferences (likes Japanese food), and allergies (peanuts), the system suggests "Teriyaki Chicken," "Sesame Spinach," and "Rice" based on that information. Furthermore, if the user selects the "Infant Menu" and specifies "Home Delivery," the meal kit will be delivered to the user's home.

[0380] In this way, the system of the present invention can provide personalized menu suggestions and meal kit delivery services tailored to the user's needs, thereby reducing the burden of daily meal preparation.

[0381] The following describes the processing flow.

[0382] Step 1:

[0383] User: Go to the account creation screen and enter your email address and password.

[0384] Terminal: Sends the entered information to the server.

[0385] Server: Registers the email address and password in the authentication system and sends a confirmation email to the user.

[0386] Step 2:

[0387] User: Receive a confirmation email and click the link to verify your account.

[0388] Terminal: Displays an authentication screen and presents a form for entering family composition, age range, preferences, allergy information, and special dietary needs.

[0389] User: Enter information such as family composition (2 adults, 1 infant), age range (30s), preferences (likes Japanese food), and allergies (peanuts) into the input form.

[0390] Terminal: Sends the entered information to the server.

[0391] Server: Stores the received personal information in the database.

[0392] Step 3:

[0393] Server: Retrieves user personal information from the database.

[0394] Server: An AI algorithm generates the optimal menu based on personal information (e.g., "Teriyaki Chicken," "Sesame-dressed Spinach," "Rice").

[0395] Server: It also takes into account the user's past menu evaluation data to generate individually optimized menus (e.g., "Miso Soup").

[0396] Terminal: Displays the suggested menu on the user's screen.

[0397] Step 4:

[0398] User: Select a special meal (e.g., "Infant Menu") from the suggested menus.

[0399] Terminal: Sends the selected option information to the server.

[0400] Server: Generates meal kits tailored to special dietary needs (e.g., "Teriyaki Chicken (seasoned for infants)", "Soft Spinach").

[0401] Step 5:

[0402] User: Select a delivery method and specify either "Home Delivery" or "Refrigerated Locker at the nearest supermarket."

[0403] Terminal: Sends the selected shipping method information to the server.

[0404] Server: Issues instructions to the logistics system and prepares for the meal kits to be delivered to the user's specified location.

[0405] Step 6:

[0406] User: Take a picture of the ingredients in your refrigerator (e.g., "cabbage, carrots, pork") and upload the image to the system.

[0407] Terminal: Sends images to the server.

[0408] Server: Performs image analysis and generates menus based on recognized ingredients (e.g., "Stir-fried pork and cabbage", "Carrot salad").

[0409] Terminal: Displays the newly generated menu on the user's screen.

[0410] (Example 1)

[0411] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0412] Conventional meal planning systems struggle to propose individually optimized menus based on users' personal information, and their suggestions tend to be biased. Furthermore, they often lack effective ways to utilize the ingredients users already possess, leading to food waste. To address these issues, there is a need for a system that generates optimal meal plans tailored to each user's individual circumstances and delivers meal kits based on those plans.

[0413] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0414] In this invention, the server includes input means for inputting the user's personal information, meal plan generation means for generating an optimal meal plan based on the personal information, presentation means for presenting the generated meal plan to the user, preparation means for preparing an ingredient kit corresponding to the selected specific food, and delivery means for delivering the ingredient kit to the user. This enables the generation of a personalized meal plan tailored to the user's individual circumstances and the delivery of an ingredient kit based on that meal plan.

[0415] "User personal information" refers to information such as the user's family structure, age group, preferences, allergy information, and special dietary needs.

[0416] "Input method" refers to an interface for users to enter personal information, including input forms and applications on a device.

[0417] "Meal plan generation method" refers to algorithms or AI models that generate optimal meal plans based on the user's personal information.

[0418] "Presentation means" refers to a method for displaying the generated meal plan to the user, and includes the terminal's display and notification system.

[0419] "Specific foods" refer to ingredients or menu items selected by users based on their special needs or requests.

[0420] "Preparation means" refers to a system for preparing ingredient kits corresponding to selected specific foods, including the selection and packing of ingredients.

[0421] "Delivery method" refers to the means of delivering prepared food kits to a location specified by the user, and includes logistics systems and delivery services.

[0422] "Images of food in storage" refers to images taken by users of food in their home refrigerators or food storage areas.

[0423] "Image analysis means" refers to a technology for analyzing food images uploaded by users and recognizing the ingredients contained within them, and includes an image recognition algorithm.

[0424] "Supplementary meal plan generation means" refers to algorithms or AI models for generating supplementary meal plans based on ingredients recognized by image analysis means.

[0425] "Auxiliary presentation means" refers to a method for displaying a meal plan generated by the auxiliary meal plan generation means to the user, and includes the terminal's display and notification system.

[0426] "Methods for using evaluation data" refers to systems and algorithms that utilize evaluation data from users' past meal plans to further optimize meal plans.

[0427] This invention is a system that provides an optimal meal plan based on the user's personal information and even delivers the ingredient kit. This system consists of three components: a server, a terminal, and the user. Specific embodiments of this system are described below.

[0428] First, the user creates an account using their device. The user enters their email address and password and registers on the account creation page. This information is sent from the device to the server, which stores it in its database. The server then sends a confirmation email to the user, and the user clicks the confirmation link in the email to authenticate their account.

[0429] Next, the user logs into their account and enters personal information such as family structure, age range, preferences, and allergy information. The entered information is sent from the device to the server and stored in the database.

[0430] The server uses an AI model to generate an optimal meal plan based on stored personal information. For example, if the family consists of two adults and one infant, prefers Japanese food, and has a peanut allergy, a menu such as "Teriyaki Chicken," "Spinach with Sesame Dressing," and "Rice" will be suggested. Because this process uses a generative AI model, the following input is used as a prompt.

[0431] Example of a prompt:

[0432] "The user's family consists of two adults and one infant, and they prefer Japanese food. They also have a peanut allergy. Please suggest the best menu based on these conditions."

[0433] The generated meal plan is sent from the server to the user's terminal, where the user can view the suggested menu on their terminal screen. The user can select specific foods from this menu (for example, infant menus), and this selection information is sent from the terminal to the server. Based on this information, the server prepares ingredient kits corresponding to the selected foods.

[0434] Furthermore, users can choose their delivery method. For example, they might have options like "home delivery" or "refrigerated locker at the nearest supermarket." The selected delivery method is then sent from the terminal to the server. The server sends instructions to the logistics system, and the meal kit is delivered to the user using the specified delivery method.

[0435] To utilize the ingredients in their storage, users take photos of the food in their refrigerator and upload the images to their device. The device sends these images to a server, which performs image analysis to recognize the ingredients. For example, if the refrigerator contains cabbage, carrots, and pork, the server will suggest supplementary menus based on these ingredients, such as "stir-fried pork and cabbage" or "carrot salad." This information is also sent from the server to the user's device.

[0436] This system supports users in their daily meal preparation and contributes to reducing food waste by generating an optimal meal plan tailored to each user's individual needs and providing delivery of ingredient kits based on that plan.

[0437] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0438] Step 1:

[0439] The user enters their email address and password on the device's registration screen and presses the "Create Account" button. This information is sent from the device to the server. The input includes an email address (e.g., example@example.com) and a password. An account creation request is sent to the server as output.

[0440] Step 2:

[0441] The server stores the received account information in a database and sends a confirmation email to the user's email address. As part of data processing, the email address and password are hashed and stored. The confirmation email is then sent to the user as output.

[0442] Example: The server stores the email address in a database and sends a confirmation email using the SMTP protocol.

[0443] Step 3:

[0444] The user clicks the link in the confirmation email to authenticate their account. This action sends the user's request back to the server, updating the account status to "Authenticated." The input is the click of the email link, and the output is the updated account authentication status.

[0445] Step 4:

[0446] After logging in, users enter personal information such as family structure, age group, preferences, and allergy information. This information is sent from the device to the server and stored in the database. Personal information is included as input, and the user profile is updated as output.

[0447] Example: The user enters information such as "2 adults, 1 infant," "likes Japanese food," and "peanut allergy."

[0448] Step 5:

[0449] The server uses an AI model to generate an optimal meal plan based on stored personal information. As part of the data processing, personal information is converted into prompt sentences and input into the AI ​​model. The generated meal plan is obtained as output.

[0450] Example prompt: "The user's family consists of two adults and one infant, and they prefer Japanese food. They have a peanut allergy. Please suggest the best menu based on these conditions."

[0451] Step 6:

[0452] The server sends the generated meal plan to the user's terminal. As output, the meal plan data is sent to the terminal.

[0453] Example: The server sends data in JSON format and converts it for display on the terminal.

[0454] Step 7:

[0455] The user reviews the suggested meal plan via their device and selects specific foods (e.g., infant menu). This selection information is sent from the device to the server as input, and then saved to the server as output.

[0456] Step 8:

[0457] The server generates information to prepare an ingredient kit corresponding to the selected specific food item. The kit contents are determined based on the specific food item information as part of the data processing. Preparation instructions are then generated as output.

[0458] Example: Modify the seasoning of "Teriyaki Chicken" to suit infants and prepare a meal kit that includes soft spinach.

[0459] Step 9:

[0460] The user selects a shipping method and sends the information from their device to the server. The input includes the selected shipping method, and the output is stored on the server.

[0461] Step 10:

[0462] The server issues instructions to the logistics system, which then delivers the food kits to the user using the specified delivery method. The data calculation generates delivery instructions for the logistics system, and the output is the execution of those instructions.

[0463] Example: A meal kit is delivered to the user's home.

[0464] Step 11:

[0465] The user takes a picture of the food in the refrigerator and uploads the image to their device. The input includes the captured image data, and the output is the image data sent to the server.

[0466] Step 12:

[0467] The server performs image analysis to recognize ingredients from the input image. As part of data processing, it analyzes the image data and extracts ingredient information. The output is a list of the recognized ingredients.

[0468] Example: Image analysis identifies cabbage, carrots, and pork.

[0469] Step 13:

[0470] The server generates a supplementary meal plan based on the recognized ingredients and sends it to the user's terminal. As part of data processing, the recognized ingredient information is input into an AI model to generate a supplementary meal plan. As output, the supplementary meal plan is generated and sent to the terminal.

[0471] Examples: "Stir-fried pork and cabbage" and "Carrot salad" are suggested.

[0472] In this way, by adding specific actions at each processing step and clearly indicating data input and output, personalized meal planning and ingredient kit delivery tailored to the user's needs can be realized.

[0473] (Application Example 1)

[0474] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0475] While conventional systems offered menu suggestions and meal kit delivery based on users' personal information, they lacked the functionality to effectively utilize ingredients in the refrigerator and propose more precisely personalized menus. Furthermore, the lack of integrated food delivery services that allowed users to directly order meals based on the suggested menus meant that user convenience was not sufficiently improved. Therefore, there is a need for a system that enables food suggestions and delivery based on individual user needs and real-time circumstances.

[0476] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0477] In this invention, the server includes input means for inputting the user's personal information, menu generation means for generating a suitable menu based on the personal information, presentation means for presenting the generated menu to the user, preparation means for preparing a meal kit corresponding to the selected special diet, delivery means for delivering the meal kit to the user, image analysis means for uploading images of ingredients in the refrigerator and analyzing those images, auxiliary menu generation means for generating auxiliary menus based on the ingredients recognized by the image analysis means, auxiliary presentation means for presenting the auxiliary menus generated by the auxiliary menu generation means to the user, and ordering means for ordering dishes based on the provided menu. As a result, the user can have an optimal menu suggested based on their personal information and the ingredients in their refrigerator, and can also directly order dishes according to that menu.

[0478] "User personal information" refers to individual information such as the user's family structure, age group, preferences, allergy information, and special dietary needs.

[0479] An "input method" is an interface for users to input personal information, and may consist of forms provided on the screen of a smartphone or computer, for example.

[0480] A "menu generation method" is a system that uses artificial intelligence (AI) to generate the optimal menu based on the user's personal information.

[0481] "Presentation means" refers to an interface for displaying the generated menu to the user, and includes the display of a smartphone or tablet.

[0482] "Preparation methods" refer to the means of preparing meal kits that correspond to the special diet selected by the user.

[0483] "Delivery method" refers to the means by which meal kits are delivered to users, including logistics systems and delivery companies.

[0484] "Image analysis means" refers to a technology for analyzing images of food items inside a refrigerator and recognizing them, and is a system that utilizes computer vision and machine learning algorithms.

[0485] The "supplementary menu generation means" is a means for generating additional menus based on ingredients recognized by the image analysis means.

[0486] The "auxiliary presentation means" is an interface for presenting menus generated by the auxiliary menu generation means to the user.

[0487] "Ordering method" refers to a means of ordering food based on the provided menu, and includes, for example, the ordering screen of a smartphone application.

[0488] This invention is a system that provides optimal menus based on the user's personal information and also delivers meal kits and food. Specific embodiments of this system are described below.

[0489] First, users create an account using a smartphone or computer application, entering their email address and password. Once the account is created, a confirmation email is sent. Users then click the link in the email to verify their account.

[0490] Next, the user enters personal information such as family structure, age group, preferences, allergy information, and special dietary needs. This information is sent from the user's device to the server and stored in the database.

[0491] The server uses AI (artificial intelligence) to generate the optimal menu based on stored personal information. For example, if the family consists of two adults and one infant, their preference is for Japanese food, and they have peanut allergies, the server will generate a menu such as "Teriyaki Chicken," "Spinach with Sesame Dressing," and "Rice."

[0492] The generated menu is presented via the user's terminal. The user can select a special meal option (e.g., a menu for infants) from the presented menu. When the user selects a special meal, that information is sent to the server, which then prepares a meal kit corresponding to that special meal. For example, a meal kit is created that includes a menu with modified seasoning for infants.

[0493] Furthermore, users take photos of the ingredients in their refrigerator and upload the images to the application. The user's device sends the images to a server, which performs image analysis. For example, if the refrigerator contains cabbage, carrots, and pork, the server recognizes these ingredients and generates suggested menus such as "stir-fried pork and cabbage" and "carrot salad."

[0494] This ensures that fresh, personalized menus are always generated and presented to the user. These supplementary menus are presented through the user's device.

[0495] Users can order food directly based on the presented menu. The order information is sent to the server, and the logistics system delivers the food to the user's specified location (home or a refrigerated locker at the nearest supermarket).

[0496] To implement this system, software libraries such as "Python," "Requests," and "PIL" are used, along with devices such as smartphones and tablets.

[0497] As a concrete example, suppose a user inputs, "Please suggest a menu using the pork, cabbage, and carrots I have in my refrigerator." In response, the AI ​​uses the following prompt:

[0498] Prompt message:

[0499] I have pork, cabbage, and carrots in the refrigerator. My family consists of two adults and one infant. They have a peanut allergy. Please suggest a menu they can enjoy.

[0500] In this way, the system of the present invention provides users with personalized menu suggestions and meal kit delivery services, thereby reducing the burden of daily meal preparation.

[0501] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0502] Step 1:

[0503] Users create an account using a smartphone or computer application, entering their email address and password. The entered information is sent from the device to the server. The server stores the user's account information in a database and generates and sends a confirmation email to the user. The user authenticates their account by clicking a link in the confirmation email. In this step, the input is the email address and password, and the output is the authenticated user account.

[0504] Step 2:

[0505] Users enter personal information into the application, such as family structure, age group, preferences, allergy information, and special dietary needs. This information is sent from the terminal to the server and stored in a database. In this step, the input is personal information, and the output is the stored personal information.

[0506] Step 3:

[0507] The server uses AI (artificial intelligence) to generate the optimal menu based on stored personal information. The AI ​​generates prompt messages that take into account the user's family structure and preferences. For example, it might generate a prompt message such as, "Please suggest the optimal menu for a user with a family structure of 2 adults and 1 infant, a preference for Japanese food, and peanut allergies," and input this into the AI ​​model. The output is an optimized menu. In this step, the input is personal information, and the output is an optimized menu.

[0508] Step 4:

[0509] The server sends the generated optimized menu to the user's terminal. The terminal displays the suggested menu to the user. The user can select special meals, etc., from the presented menu. In this step, the input is the optimized menu, and the output is the menu displayed on the user's terminal.

[0510] Step 5:

[0511] The user takes a picture of the food in their refrigerator with their camera and uploads the image to the application. The image is sent from the device to the server. The server uses image analysis technology to recognize the food in the image. In this step, the input is an image of the food in the refrigerator, and the output is data of the recognized food.

[0512] Step 6:

[0513] The server generates a suggested menu based on the recognized ingredient data. For example, it generates a prompt such as, "There is pork, cabbage, and carrots in the refrigerator. Please suggest a menu using these ingredients," and inputs it into the AI ​​model. The output is a suggested menu. In this step, the input is the recognized ingredient data, and the output is a suggested menu.

[0514] Step 7:

[0515] The server sends the generated supplementary menu to the user's terminal. The terminal displays the suggested supplementary menu to the user. In this step, the input is the supplementary menu, and the output is the supplementary menu displayed on the user's terminal.

[0516] Step 8:

[0517] Users can order dishes based on the presented menu and supplementary menus. Order information is sent from the terminal to the server, which prepares the dishes via a logistics system and delivers them to the user's specified location. In this step, the input is the order information, and the output is the dishes delivered to the user's specified location.

[0518] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0519] This invention is a system that provides an optimal menu considering not only the user's family structure, preferences, and health condition, but also the user's emotions, and even delivers the meal kit. Specific embodiments of this system are described below.

[0520] First, the user creates an account. The user uses their device to enter their email address and password to create the account. Afterward, the user receives a confirmation email and clicks the link to authenticate their account. Next, they enter personal information such as family structure, age group, preferences, allergy information, and special dietary needs. This information is sent from the user's device to the server and stored in the database.

[0521] Next, the server uses AI to generate the optimal menu based on the stored personal information. For example, if the family consists of two adults and one infant, and their preference is for Japanese food, a menu such as "Teriyaki Chicken," "Spinach with Sesame Dressing," and "Rice" will be generated. The server also takes into account the user's past menu evaluation data to present an even more individually optimized menu.

[0522] The generated menu is presented to the user via a terminal. The user can select a special meal (e.g., a menu for infants) from the presented menu. When the user selects a special meal, that information is sent to the server, which generates a meal kit corresponding to the special meal. For example, for a family with an infant, a meal kit is prepared that includes teriyaki chicken seasoned for infants and soft spinach.

[0523] Next, the user can choose a delivery method, such as "home delivery" or "refrigerated locker at the nearest supermarket." If the user selects "home delivery," that information is sent to the server, and instructions are sent to the logistics system. As a result, the meal kit is delivered to the user's home.

[0524] Furthermore, users can take photos of the ingredients in their refrigerator and upload the images to the system. The device sends the images to the server, which then performs image analysis. For example, if the refrigerator contains cabbage, carrots, and pork, the server recognizes these ingredients and generates menus such as "stir-fried pork and cabbage" and "carrot salad." The generated menus are then presented to the user via the device.

[0525] Furthermore, this system includes an emotion engine that recognizes the user's emotions. Users can communicate their current mood and feelings to the system using an emotion input interface. For example, if a user inputs an emotion such as "I'm feeling stressed," the server will adjust the menu based on that information. For instance, it can suggest menus using ingredients with relaxing effects or recommend the user's favorite comfort foods.

[0526] The emotion engine continuously collects user emotional data and uses it for long-term preference analysis. This allows it to learn what emotions users have felt and which menu items they have preferred in the past, enabling it to provide even more personalized suggestions.

[0527] For example, if a user enters their family structure (2 adults, 1 infant), preferences (likes Japanese food), and allergies (peanuts), the system will suggest "Teriyaki Chicken," "Sesame Spinach," and "Rice." Furthermore, if the user selects the "Infant Menu" and specifies "Home Delivery," the meal kit will be delivered to the user's home. Also, if the user enters that they are "feeling stressed," the system will suggest additional items such as a "Matcha-flavored dessert" that has a relaxing effect.

[0528] In this way, the system of the present invention provides personalized menu suggestions and meal kit delivery services tailored to the user's needs and emotions, thereby reducing the burden of daily meal preparation and improving the quality of life.

[0529] The following describes the processing flow.

[0530] Step 1:

[0531] User: Go to the account creation screen and enter your email address and password.

[0532] Terminal: Sends the entered information to the server.

[0533] Server: Registers the email address and password in the authentication system and sends a confirmation email to the user.

[0534] Step 2:

[0535] User: Receive a confirmation email and click the link to verify your account.

[0536] Terminal: Displays an authentication screen and presents a form for entering family composition, age range, preferences, allergy information, and special dietary needs.

[0537] User: Enter information such as family composition (2 adults, 1 infant), age range (30s), preferences (likes Japanese food), and allergies (peanuts) into the input form.

[0538] Terminal: Sends the entered information to the server.

[0539] Server: Stores the received personal information in the database.

[0540] Step 3:

[0541] Server: Retrieves user personal information from the database.

[0542] Server: An AI algorithm generates the optimal menu based on personal information (e.g., "Teriyaki Chicken," "Sesame-dressed Spinach," "Rice").

[0543] Server: It also takes into account the user's past menu evaluation data to generate individually optimized menus (e.g., "Miso Soup").

[0544] Terminal: Displays the suggested menu on the user's screen.

[0545] Step 4:

[0546] User: Select a special meal (e.g., "Infant Menu") from the suggested menus.

[0547] Terminal: Sends the selected option information to the server.

[0548] Server: Generates meal kits tailored to special dietary needs (e.g., "Teriyaki Chicken (seasoned for infants)", "Soft Spinach").

[0549] Step 5:

[0550] User: Select a delivery method and specify either "Home Delivery" or "Refrigerated Locker at the nearest supermarket."

[0551] Terminal: Sends the selected shipping method information to the server.

[0552] Server: Issues instructions to the logistics system and prepares for the meal kits to be delivered to the user's specified location.

[0553] Step 6:

[0554] User: Take a picture of the ingredients in your refrigerator (e.g., "cabbage, carrots, pork") and upload the image to the system.

[0555] Terminal: Sends images to the server.

[0556] Server: Performs image analysis and generates menus based on recognized ingredients (e.g., "Stir-fried pork and cabbage", "Carrot salad").

[0557] Terminal: Displays the newly generated menu on the user's screen.

[0558] Step 7:

[0559] User: Use the emotion input interface to enter your current mood and emotions into the system (e.g., "I'm feeling stressed").

[0560] Terminal: Sends emotional data to the server.

[0561] Server: The emotion engine analyzes emotional data and adjusts the menu based on emotions (e.g., a relaxing "matcha dessert").

[0562] Terminal: Displays the adjusted new menu on the user's screen.

[0563] (Example 2)

[0564] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0565] Conventional menu generation systems suggest menus based on users' personal information and preferences, but they are insufficient in generating personalized menus that take into account users' emotions and refrigerator inventory, and in properly managing and delivering them. Furthermore, they lack sufficient functionality to optimize menus over the long term by utilizing users' past menu evaluations and emotional data, making it difficult to continuously increase user satisfaction.

[0566] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0567] In this invention, the server includes input means for inputting the user's personal information, menu generation means for generating a suitable menu based on the personal information and the user's emotional data, presentation means for presenting the generated menu to the user, preparation means for preparing a meal kit corresponding to a special meal selected by the user, delivery means for delivering the meal kit to the user, image analysis means for analyzing images of ingredients in the refrigerator uploaded by the user, auxiliary menu generation means for generating a menu based on the ingredients recognized by the image analysis means, auxiliary presentation means for presenting the generated auxiliary menu to the user, and evaluation data utilization means for further optimizing the menu based on the user's past menu evaluation data and emotional data. This makes it possible to provide and deliver optimized and personalized menus, and meal kits based on them, taking into account the user's personal information, emotional data, and the inventory in the refrigerator.

[0568] - A "user" refers to an individual who uses this system to create an account and provides personal information and sentiment data.

[0569] "Personal information" refers to information about the user, such as family structure, age group, preferences, allergy information, and special dietary needs.

[0570] "Emotional data" refers to information that users input, indicating their current mood and emotions.

[0571] "Menu generation means" refers to a function that generates appropriate menus based on personal information and emotional data.

[0572] "Presentation means" refers to a function for notifying or displaying the generated menu to the user.

[0573] "Special meals" refer to meal menus tailored to specific needs selected by the user.

[0574] "Preparation means" refers to the function of preparing meal kits that accommodate special dietary needs.

[0575] "Delivery method" refers to the function for delivering prepared meal kits to users.

[0576] "Image analysis means" refers to a function that analyzes images of food items in a refrigerator uploaded by the user to recognize the food items.

[0577] "Auxiliary menu generation means" refers to a function for generating menus based on ingredients recognized by image analysis means.

[0578] "Auxiliary presentation means" refers to a function for notifying or displaying the generated auxiliary menu to the user.

[0579] "Means of using evaluation data" refers to functions that further optimize menus based on the user's past menu evaluation data and sentiment data.

[0580] A "meal kit" refers to a set of ingredients prepared based on a pre-designed menu.

[0581] "Food in the refrigerator" refers to the food stored in the user's refrigerator.

[0582] This invention is a system that provides an optimal menu considering the user's personal information and emotional data, and even delivers the meal kit. Specifically, the system operates in the following steps.

[0583] First, the user accesses the system's webpage or app using their device. The user enters their email address and password to create an account. The server receives the entered information, registers the new user information in the database, and then sends a confirmation email. The user clicks the link in the email to authenticate their account. This completes the account creation process.

[0584] Next, the user enters personal information from their device, such as family structure, preferences, allergy information, and special diet needs. This information is sent to the server and stored in the database. For example, if the family consists of two adults and one infant, preferences are for Japanese food, and allergies include peanuts, this data will be stored in the user's profile.

[0585] The server uses stored personal information to call up an AI model (such as OpenAI's GPT-4) to generate an optimal menu. For example, if a family consists of two adults and one infant and prefers Japanese food, the AI ​​will suggest a menu such as "teriyaki chicken," "spinach with sesame dressing," and "rice." The server saves the generated menu in a database and presents it to the user via the terminal.

[0586] Users can review the menu and select special meals (e.g., infant menus). The selected information is sent to the server, which then prepares a meal kit tailored to the special meal. For example, the server might prepare a meal kit of "Teriyaki Chicken" with the seasoning modified for infants and include "Soft Spinach."

[0587] Next, the user selects a delivery method using a terminal. Options include "home delivery" and "refrigerated locker at the nearest supermarket." If the user selects "home delivery," that information is sent to the server. The server then instructs the logistics system to deliver the meal kit to the location specified by the user.

[0588] Furthermore, the user takes pictures of the food in the refrigerator with their device's camera and uploads the images to the server. The server uses an image analysis engine (such as the Google Cloud Vision API) to recognize the food. Based on the recognized food, the server uses an auxiliary menu generation method to generate the optimal menu and presents it to the user. For example, if the refrigerator contains cabbage, carrots, and pork, menus such as "stir-fried pork and cabbage" and "carrot salad" will be suggested.

[0589] Users input their current mood and emotions through their device. For example, they might input "I'm feeling stressed" or "I'm tired." The server adjusts the menu based on the emotional input data. For instance, if the user is feeling stressed, a relaxing dessert such as a "matcha-flavored dessert" might be suggested.

[0590] The server continuously collects user emotional data and provides even more personalized suggestions based on long-term preference analysis. For example, if a user previously preferred a particular menu item when they were feeling stressed, this information is used to optimize future suggestions.

[0591] Specific example:

[0592] The user enters their family composition (2 adults, 1 infant), preferences (Japanese food), and allergies (peanuts) into the terminal. The server receives this information, saves it to a database, and then calls an AI model to generate menus such as "Teriyaki Chicken," "Spinach with Sesame Dressing," and "Rice." The user reviews the suggestions on the terminal and selects a special meal (for the infant). Based on this information, the server prepares a meal kit, and if the user selects "Home Delivery," the meal kit is delivered. Additionally, if the user enters "I'm feeling stressed," the server will suggest additional menu items such as a relaxing "Matcha-flavored dessert."

[0593] Example of a prompt:

[0594] The user has entered their family structure (2 adults, 1 infant) and preferred Japanese food. Peanut allergy information has also been provided. Please suggest the most suitable menu and delivery options.

[0595] Thus, the system of the present invention provides personalized menu suggestions and meal kit delivery services tailored to the user's needs and emotions, thereby reducing the burden of daily meal preparation and improving the quality of life.

[0596] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0597] Step 1:

[0598] User account creation

[0599] The user accesses the system's webpage or app using their device. The user enters their email address and password and submits the account creation form. The server receives this information and registers the new user information in its database. The server sends a confirmation email to the registered email address. The user clicks the link in the email to authenticate their account, completing the account creation process.

[0600] Input: User's email address, password

[0601] Output: Account creation confirmation email

[0602] Step 2:

[0603] Input and storage of user personal information

[0604] Users use their devices to input personal information such as family structure, preferences, allergy information, and special dietary needs. The entered information is sent to the server, which stores it in a database. For example, if a user inputs that their family consists of two adults and one infant, their preferences are for Japanese food, and their allergy is peanuts, that information will be saved in the user's profile.

[0605] Input: Family composition, preferences, allergy information, special diet needs

[0606] Output: Personal information stored in the database

[0607] Step 3:

[0608] AI-powered optimal menu generation

[0609] The server uses stored personal information to call the appropriate AI model (e.g., a generative AI model). The AI ​​generates an optimal menu based on the personal information and emotional data. For example, if the family consists of two adults and one infant, and their preference is for Japanese food, the AI ​​will suggest a menu such as "teriyaki chicken," "spinach with sesame dressing," and "rice." The server saves the generated menu in a database.

[0610] Input: Personal Information

[0611] Output: Optimal menu

[0612] Step 4:

[0613] User-selected menu

[0614] The user reviews the suggested menu through their device. If a special diet (e.g., infant menu) is selected, the selection information is sent to the server, which stores it in a database. The process of generating a meal kit based on the special diet information is then initiated.

[0615] Input: Suggested menu, selection of special meals

[0616] Output: Special diet information stored on the server

[0617] Step 5:

[0618] Production of meal kits based on special diets

[0619] The server generates optimized meal kits based on the user's selected special diet information. For example, it might prepare a meal kit for infants with modified seasonings for "teriyaki chicken" and include "soft spinach." This information is managed by the server and used to integrate with the logistics system.

[0620] Input: Special diet information

[0621] Output: Optimized meal kit information

[0622] Step 6:

[0623] User selection of delivery method and delivery of meal kits

[0624] The user selects a delivery method using a terminal. Options include "home delivery" and "refrigerated locker at the nearest supermarket." If the user selects "home delivery," that information is sent to the server. The server then instructs the logistics system, and the meal kit is delivered to the location specified by the user.

[0625] Input: Select shipping method

[0626] Output: Delivery Instructions

[0627] Step 7:

[0628] Image recognition of food items inside a refrigerator

[0629] The user takes a picture of the food in their refrigerator with their camera and uploads the image from their device to the server. The server uses an image analysis engine (e.g., an image analysis API) to recognize the food in the image. Based on the recognized food, the server generates an optimal menu and presents it to the user.

[0630] Input: Image of food items in the refrigerator

[0631] Output: Optimal menu

[0632] Step 8:

[0633] User emotional input and menu adjustment

[0634] Users input their current mood and emotions through their device, such as "I'm stressed" or "I'm tired." The device sends this information to a server. The server adjusts the menu based on the emotional input data. For example, if the user is feeling stressed, a relaxing dessert such as a "matcha-flavored dessert" might be suggested.

[0635] Input: Current mood or feelings

[0636] Output: Adjusted menu

[0637] Step 9:

[0638] Accumulation of emotional data and long-term preference analysis

[0639] The server continuously accumulates user emotional data. This stored emotional data is used for long-term preference analysis. This allows the system to learn what emotions users have experienced and which menu items they have preferred in the past, enabling more personalized suggestions. For example, if a user has preferred a particular menu item when feeling stressed in the past, this information can be used to optimize future suggestions.

[0640] Input: Sentiment data

[0641] Output: Personalized suggestions

[0642] (Application Example 2)

[0643] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0644] In modern dietary habits, it is difficult for users to create personalized menus tailored to their family structure, preferences, health status, and even their emotions, and then quickly prepare meal kits based on those menus. Furthermore, there is no consistent system for efficiently utilizing ingredients in the refrigerator while providing meals that meet individual needs. As a result, users often experience stress and inconvenience.

[0645] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes an input means for inputting the user's personal information, a menu generation means for generating a suitable menu based on the personal information, a presentation means for presenting the generated menu to the user, a preparation means for preparing a meal kit corresponding to the selected special diet, a delivery means for delivering the meal kit to the user, an emotion input means for inputting or recognizing the user's emotions, and an emotion adjustment means for adjusting the menu based on the emotion input means. This enables personalized menu suggestions that respond to the user's individual needs and emotions, as well as efficient delivery of meal kits.

[0646] "User personal information" refers to information related to an individual user, such as family structure, age group, preferences, allergy information, and special dietary needs.

[0647] An "input method" refers to an interface or device that allows a user to input personal information or emotional data into a system.

[0648] A "menu generation method" is a means for generating an optimal menu based on the user's personal information and emotional data.

[0649] "Presentation means" refers to means of presenting the generated menu to the user visually or audibly.

[0650] "Preparation methods" refer to the means of preparing meal kits that correspond to the selected special diet.

[0651] "Delivery method" refers to the means by which prepared meal kits are delivered to users.

[0652] An "emotion input method" is a means of inputting or recognizing a user's emotions.

[0653] "Emotion adjustment methods" refer to methods for adjusting menus based on the user's emotional data.

[0654] "Image analysis means" refers to a means for analyzing images of food items inside a refrigerator that have been uploaded by a user.

[0655] The "supplementary menu generation means" is a means for generating supplementary menus based on ingredients recognized by the image analysis means.

[0656] "Auxiliary presentation means" refers to a means for presenting menus generated by the auxiliary menu generation means to the user.

[0657] This invention is a system that generates menus and provides / delivers meal kits, taking into account the user's family structure, preferences, health status, and emotions. The system operates when the user creates an account via smartphone, enters personal information, uploads images of ingredients in their refrigerator, or enters their emotions.

[0658] First, users create an account using their smartphone. They enter their email address and password to create the account. Next, they enter personal information such as family structure, age group, preferences, allergy information, and special dietary needs. This information is sent from the device to the server and stored in the database.

[0659] After personal information is saved, the server uses AI to generate an optimal menu based on the saved information. For example, if the family consists of two adults and one infant, and their preference is for Japanese food, a menu such as "Teriyaki Chicken," "Spinach with Sesame Dressing," and "Rice" will be generated. Furthermore, the menu is adjusted using an emotion input method that allows the user to input or recognize their emotions. If the user inputs that they are feeling stressed, the menu will suggest dishes using ingredients with relaxing effects or their favorite comfort foods.

[0660] The generated menu is presented to the user via a terminal. The user can select a special meal (for example, a menu for infants) from the presented menu. When the user selects a special meal, that information is sent to the server, which generates a meal kit corresponding to the special meal. For example, for a family with an infant, a meal kit is prepared that includes teriyaki chicken seasoned for infants and soft spinach. Next, the user selects a delivery method and chooses "home delivery," at which point the meal kit is delivered to the user's home.

[0661] Furthermore, users can take photos of the ingredients in their refrigerator and upload the images to the system. The device sends the images to the server, which then performs image analysis. For example, if the refrigerator contains cabbage, carrots, and pork, the server recognizes these ingredients and generates menus such as "stir-fried pork and cabbage" and "carrot salad." The generated menus are then presented to the user via the device.

[0662] The hardware and software used include user terminals (e.g., smartphones), image recognition technology (e.g., OpenCV, TensorFlow), and emotion recognition technology (e.g., EmotionRecognition model). Menus are generated using AI, and deep learning models (e.g., TensorFlow or PyTorch) are used on the server side.

[0663] For example, if a user enters their family structure (2 adults, 1 infant), preferences (likes Japanese food), and allergies (peanuts), the system will suggest "Teriyaki Chicken," "Sesame Spinach," and "Rice." Furthermore, if the user selects the "Infant Menu" and specifies "Home Delivery," the meal kit will be delivered to the user's home. Also, if the user enters that they are "feeling stressed," the system will suggest additional items such as a "Matcha-flavored dessert" that has a relaxing effect.

[0664] Examples of specific prompt statements include the following:

[0665] "Based on the following profile and emotional data, please suggest the following menu: User Profile: Family Composition: 2 adults, 1 infant; Preferences: Japanese food; Allergies: Peanuts; Special Diet Needs: Infant menu; Emotions: Feeling stressed."

[0666] By fulfilling these requirements, the system can provide personalized menu suggestions and meal kit delivery services tailored to the user's needs and emotions.

[0667] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0668] Step 1:

[0669] The user creates an account using their smartphone. The input consists of an email address and password, which the device receives and sends to the server. The server receives this information, registers the user account in its database, and sends a confirmation email. The output is a confirmation email indicating successful account creation.

[0670] Step 2:

[0671] The user enters personal information such as family structure, age group, preferences, allergy information, and special dietary needs. This information is sent from the terminal to the server, which stores this data in a database. The output confirms the saving of the personal information.

[0672] Step 3:

[0673] The server uses AI to generate the optimal menu based on stored personal information. In this process, personal information stored in a database is used as input, and a generation AI model is used to calculate and generate the menu. The output is the generated menu data.

[0674] Step 4:

[0675] The terminal displays the generated menu to the user. The input is menu data received from the server, which the terminal displays to the user. The output is the menu presented to the user.

[0676] Step 5:

[0677] The user selects a special meal from the presented menu. The user's selection is sent from the terminal to the server, which verifies this information. The input is the user's selected special meal information, and the server prepares the meal kit based on it. The output is the data for the prepared meal kit.

[0678] Step 6:

[0679] The user selects a shipping method. The user's selected shipping method is sent from the terminal to the server, which then issues instructions to the logistics system. The input is the user's selected shipping method, and the output is a notification that the shipping arrangements have been completed.

[0680] Step 7:

[0681] The user takes a picture of the food in their refrigerator and uploads the image from their device to the server. The server uses image analysis to recognize the food. The input is an image of the food in the refrigerator, and the output is the analyzed food data.

[0682] Step 8:

[0683] The server generates supplementary menus based on recognized ingredient data. In this process, the analyzed ingredient data is used as input, and a generation AI model is used to calculate and generate the supplementary menus. The output is the data of the generated supplementary menus.

[0684] Step 9:

[0685] The terminal displays the generated supplementary menu to the user. The input is the supplementary menu data received from the server, which the terminal displays to the user. The output is the supplementary menu presented to the user.

[0686] Step 10:

[0687] The user inputs an emotion or uploads a facial image. The device sends this data to the server, which uses the emotion input method to recognize the emotion. The input is the user's emotion or facial image data, and the output is the recognized emotion data.

[0688] Step 11:

[0689] The server adjusts the menu based on recognized emotion data. This process uses emotion data as input and readjusts the menu using emotion adjustment mechanisms. The output is the adjusted menu data.

[0690] This processing flow enables the delivery of meal kits and menu suggestions optimized to the individual needs and emotions of each user.

[0691] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0692] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0693] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0694] [Third Embodiment]

[0695] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0696] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0697] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0698] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0699] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0700] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0701] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0702] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0703] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0704] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0705] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0706] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0707] This invention is a system that provides an optimal menu based on the user's personal information, such as family structure, preferences, and health status, and even delivers the meal kit. Specific embodiments of this system are described below.

[0708] First, the user creates an account. The user uses their device to enter their email address and password to create the account. Afterward, the user receives a confirmation email and clicks the link to authenticate their account. Next, they enter personal information such as family structure, age group, preferences, allergy information, and special dietary needs. This information is sent from the user's device to the server and stored in the database.

[0709] Next, the server uses AI to generate the optimal menu based on the stored personal information. For example, if the family consists of two adults and one infant, and their preference is for Japanese food, a menu such as "Teriyaki Chicken," "Spinach with Sesame Dressing," and "Rice" will be generated. The server also takes into account the user's past menu evaluation data to present an even more individually optimized menu.

[0710] The generated menu is presented to the user via a terminal. The user can select a special meal (e.g., a menu for infants) from the presented menu. When the user selects a special meal, that information is sent to the server, which generates a meal kit corresponding to the special meal. For example, for a family with an infant, a meal kit is prepared that includes teriyaki chicken seasoned for infants and soft spinach.

[0711] Next, the user can choose a delivery method, such as "home delivery" or "refrigerated locker at the nearest supermarket." If the user selects "home delivery," that information is sent to the server, and instructions are sent to the logistics system. As a result, the meal kit is delivered to the user's home.

[0712] Furthermore, users can take photos of the ingredients in their refrigerator and upload the images to the system. The device sends the images to the server, which then performs image analysis. For example, if the refrigerator contains cabbage, carrots, and pork, the server recognizes these ingredients and generates menus such as "stir-fried pork and cabbage" and "carrot salad." The generated menus are then presented to the user via the device.

[0713] As a concrete example, the following process can be considered: When the user enters their family composition (2 adults, 1 infant), preferences (likes Japanese food), and allergies (peanuts), the system suggests "Teriyaki Chicken," "Sesame Spinach," and "Rice" based on that information. Furthermore, if the user selects the "Infant Menu" and specifies "Home Delivery," the meal kit will be delivered to the user's home.

[0714] In this way, the system of the present invention can provide personalized menu suggestions and meal kit delivery services tailored to the user's needs, thereby reducing the burden of daily meal preparation.

[0715] The following describes the processing flow.

[0716] Step 1:

[0717] User: Go to the account creation screen and enter your email address and password.

[0718] Terminal: Sends the entered information to the server.

[0719] Server: Registers the email address and password in the authentication system and sends a confirmation email to the user.

[0720] Step 2:

[0721] User: Receive a confirmation email and click the link to verify your account.

[0722] Terminal: Displays an authentication screen and presents a form for entering family composition, age range, preferences, allergy information, and special dietary needs.

[0723] User: Enter information such as family composition (2 adults, 1 infant), age range (30s), preferences (likes Japanese food), and allergies (peanuts) into the input form.

[0724] Terminal: Sends the entered information to the server.

[0725] Server: Stores the received personal information in the database.

[0726] Step 3:

[0727] Server: Retrieves user personal information from the database.

[0728] Server: An AI algorithm generates the optimal menu based on personal information (e.g., "Teriyaki Chicken," "Sesame-dressed Spinach," "Rice").

[0729] Server: It also takes into account the user's past menu evaluation data to generate individually optimized menus (e.g., "Miso Soup").

[0730] Terminal: Displays the suggested menu on the user's screen.

[0731] Step 4:

[0732] User: Select a special meal (e.g., "Infant Menu") from the suggested menus.

[0733] Terminal: Sends the selected option information to the server.

[0734] Server: Generates meal kits tailored to special dietary needs (e.g., "Teriyaki Chicken (seasoned for infants)", "Soft Spinach").

[0735] Step 5:

[0736] User: Select a delivery method and specify either "Home Delivery" or "Refrigerated Locker at the nearest supermarket."

[0737] Terminal: Sends the selected shipping method information to the server.

[0738] Server: Issues instructions to the logistics system and prepares for the meal kits to be delivered to the user's specified location.

[0739] Step 6:

[0740] User: Take a picture of the ingredients in your refrigerator (e.g., "cabbage, carrots, pork") and upload the image to the system.

[0741] Terminal: Sends images to the server.

[0742] Server: Performs image analysis and generates menus based on recognized ingredients (e.g., "Stir-fried pork and cabbage", "Carrot salad").

[0743] Terminal: Displays the newly generated menu on the user's screen.

[0744] (Example 1)

[0745] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0746] Conventional meal planning systems struggle to propose individually optimized menus based on users' personal information, and their suggestions tend to be biased. Furthermore, they often lack effective ways to utilize the ingredients users already possess, leading to food waste. To address these issues, there is a need for a system that generates optimal meal plans tailored to each user's individual circumstances and delivers meal kits based on those plans.

[0747] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0748] In this invention, the server includes input means for inputting the user's personal information, meal plan generation means for generating an optimal meal plan based on the personal information, presentation means for presenting the generated meal plan to the user, preparation means for preparing an ingredient kit corresponding to the selected specific food, and delivery means for delivering the ingredient kit to the user. This enables the generation of a personalized meal plan tailored to the user's individual circumstances and the delivery of an ingredient kit based on that meal plan.

[0749] "User personal information" refers to information such as the user's family structure, age group, preferences, allergy information, and special dietary needs.

[0750] "Input method" refers to an interface for users to enter personal information, including input forms and applications on a device.

[0751] "Meal plan generation method" refers to algorithms or AI models that generate optimal meal plans based on the user's personal information.

[0752] "Presentation means" refers to a method for displaying the generated meal plan to the user, and includes the terminal's display and notification system.

[0753] "Specific foods" refer to ingredients or menu items selected by users based on their special needs or requests.

[0754] "Preparation means" refers to a system for preparing ingredient kits corresponding to selected specific foods, including the selection and packing of ingredients.

[0755] "Delivery method" refers to the means of delivering prepared food kits to a location specified by the user, and includes logistics systems and delivery services.

[0756] "Images of food in storage" refers to images taken by users of food in their home refrigerators or food storage areas.

[0757] "Image analysis means" refers to a technology for analyzing food images uploaded by users and recognizing the ingredients contained within them, and includes an image recognition algorithm.

[0758] "Supplementary meal plan generation means" refers to algorithms or AI models for generating supplementary meal plans based on ingredients recognized by image analysis means.

[0759] "Auxiliary presentation means" refers to a method for displaying a meal plan generated by the auxiliary meal plan generation means to the user, and includes the terminal's display and notification system.

[0760] "Methods for using evaluation data" refers to systems and algorithms that utilize evaluation data from users' past meal plans to further optimize meal plans.

[0761] This invention is a system that provides an optimal meal plan based on the user's personal information and even delivers the ingredient kit. This system consists of three components: a server, a terminal, and the user. Specific embodiments of this system are described below.

[0762] First, the user creates an account using their device. The user enters their email address and password and registers on the account creation page. This information is sent from the device to the server, which stores it in its database. The server then sends a confirmation email to the user, and the user clicks the confirmation link in the email to authenticate their account.

[0763] Next, the user logs into their account and enters personal information such as family structure, age range, preferences, and allergy information. The entered information is sent from the device to the server and stored in the database.

[0764] The server uses an AI model to generate an optimal meal plan based on stored personal information. For example, if the family consists of two adults and one infant, prefers Japanese food, and has a peanut allergy, a menu such as "Teriyaki Chicken," "Spinach with Sesame Dressing," and "Rice" will be suggested. Because this process uses a generative AI model, the following input is used as a prompt.

[0765] Example of a prompt:

[0766] "The user's family consists of two adults and one infant, and they prefer Japanese food. They also have a peanut allergy. Please suggest the best menu based on these conditions."

[0767] The generated meal plan is sent from the server to the user's terminal, where the user can view the suggested menu on their terminal screen. The user can select specific foods from this menu (for example, infant menus), and this selection information is sent from the terminal to the server. Based on this information, the server prepares ingredient kits corresponding to the selected foods.

[0768] Furthermore, users can choose their delivery method. For example, they might have options like "home delivery" or "refrigerated locker at the nearest supermarket." The selected delivery method is then sent from the terminal to the server. The server sends instructions to the logistics system, and the meal kit is delivered to the user using the specified delivery method.

[0769] To utilize the ingredients in their storage, users take photos of the food in their refrigerator and upload the images to their device. The device sends these images to a server, which performs image analysis to recognize the ingredients. For example, if the refrigerator contains cabbage, carrots, and pork, the server will suggest supplementary menus based on these ingredients, such as "stir-fried pork and cabbage" or "carrot salad." This information is also sent from the server to the user's device.

[0770] This system supports users in their daily meal preparation and contributes to reducing food waste by generating an optimal meal plan tailored to each user's individual needs and providing delivery of ingredient kits based on that plan.

[0771] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0772] Step 1:

[0773] The user enters their email address and password on the device's registration screen and presses the "Create Account" button. This information is sent from the device to the server. The input includes an email address (e.g., example@example.com) and a password. An account creation request is sent to the server as output.

[0774] Step 2:

[0775] The server stores the received account information in a database and sends a confirmation email to the user's email address. As part of data processing, the email address and password are hashed and stored. The confirmation email is then sent to the user as output.

[0776] Example: The server stores the email address in a database and sends a confirmation email using the SMTP protocol.

[0777] Step 3:

[0778] The user clicks the link in the confirmation email to authenticate their account. This action sends the user's request back to the server, updating the account status to "Authenticated." The input is the click of the email link, and the output is the updated account authentication status.

[0779] Step 4:

[0780] After logging in, users enter personal information such as family structure, age group, preferences, and allergy information. This information is sent from the device to the server and stored in the database. Personal information is included as input, and the user profile is updated as output.

[0781] Example: The user enters information such as "2 adults, 1 infant," "likes Japanese food," and "peanut allergy."

[0782] Step 5:

[0783] The server uses an AI model to generate an optimal meal plan based on stored personal information. As part of the data processing, personal information is converted into prompt sentences and input into the AI ​​model. The generated meal plan is obtained as output.

[0784] Example prompt: "The user's family consists of two adults and one infant, and they prefer Japanese food. They have a peanut allergy. Please suggest the best menu based on these conditions."

[0785] Step 6:

[0786] The server sends the generated meal plan to the user's terminal. As output, the meal plan data is sent to the terminal.

[0787] Example: The server sends data in JSON format and converts it for display on the terminal.

[0788] Step 7:

[0789] The user reviews the suggested meal plan via their device and selects specific foods (e.g., infant menu). This selection information is sent from the device to the server as input, and then saved to the server as output.

[0790] Step 8:

[0791] The server generates information to prepare an ingredient kit corresponding to the selected specific food item. The kit contents are determined based on the specific food item information as part of the data processing. Preparation instructions are then generated as output.

[0792] Example: Modify the seasoning of "Teriyaki Chicken" to suit infants and prepare a meal kit that includes soft spinach.

[0793] Step 9:

[0794] The user selects a shipping method and sends the information from their device to the server. The input includes the selected shipping method, and the output is stored on the server.

[0795] Step 10:

[0796] The server issues instructions to the logistics system, which then delivers the food kits to the user using the specified delivery method. The data calculation generates delivery instructions for the logistics system, and the output is the execution of those instructions.

[0797] Example: A meal kit is delivered to the user's home.

[0798] Step 11:

[0799] The user takes a picture of the food in the refrigerator and uploads the image to their device. The input includes the captured image data, and the output is the image data sent to the server.

[0800] Step 12:

[0801] The server performs image analysis to recognize ingredients from the input image. As part of data processing, it analyzes the image data and extracts ingredient information. The output is a list of the recognized ingredients.

[0802] Example: Image analysis identifies cabbage, carrots, and pork.

[0803] Step 13:

[0804] The server generates a supplementary meal plan based on the recognized ingredients and sends it to the user's terminal. As part of data processing, the recognized ingredient information is input into an AI model to generate a supplementary meal plan. As output, the supplementary meal plan is generated and sent to the terminal.

[0805] Examples: "Stir-fried pork and cabbage" and "Carrot salad" are suggested.

[0806] In this way, by adding specific actions at each processing step and clearly indicating data input and output, personalized meal planning and ingredient kit delivery tailored to the user's needs can be realized.

[0807] (Application Example 1)

[0808] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0809] While conventional systems offered menu suggestions and meal kit delivery based on users' personal information, they lacked the functionality to effectively utilize ingredients in the refrigerator and propose more precisely personalized menus. Furthermore, the lack of integrated food delivery services that allowed users to directly order meals based on the suggested menus meant that user convenience was not sufficiently improved. Therefore, there is a need for a system that enables food suggestions and delivery based on individual user needs and real-time circumstances.

[0810] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0811] In this invention, the server includes input means for inputting the user's personal information, menu generation means for generating a suitable menu based on the personal information, presentation means for presenting the generated menu to the user, preparation means for preparing a meal kit corresponding to the selected special diet, delivery means for delivering the meal kit to the user, image analysis means for uploading images of ingredients in the refrigerator and analyzing those images, auxiliary menu generation means for generating auxiliary menus based on the ingredients recognized by the image analysis means, auxiliary presentation means for presenting the auxiliary menus generated by the auxiliary menu generation means to the user, and ordering means for ordering dishes based on the provided menu. As a result, the user can have an optimal menu suggested based on their personal information and the ingredients in their refrigerator, and can also directly order dishes according to that menu.

[0812] "User personal information" refers to individual information such as the user's family structure, age group, preferences, allergy information, and special dietary needs.

[0813] An "input method" is an interface for users to input personal information, and may consist of forms provided on the screen of a smartphone or computer, for example.

[0814] A "menu generation method" is a system that uses artificial intelligence (AI) to generate the optimal menu based on the user's personal information.

[0815] "Presentation means" refers to an interface for displaying the generated menu to the user, and includes the display of a smartphone or tablet.

[0816] "Preparation methods" refer to the means of preparing meal kits that correspond to the special diet selected by the user.

[0817] "Delivery method" refers to the means by which meal kits are delivered to users, including logistics systems and delivery companies.

[0818] "Image analysis means" refers to a technology for analyzing images of food items inside a refrigerator and recognizing them, and is a system that utilizes computer vision and machine learning algorithms.

[0819] The "supplementary menu generation means" is a means for generating additional menus based on ingredients recognized by the image analysis means.

[0820] The "auxiliary presentation means" is an interface for presenting menus generated by the auxiliary menu generation means to the user.

[0821] "Ordering method" refers to a means of ordering food based on the provided menu, and includes, for example, the ordering screen of a smartphone application.

[0822] This invention is a system that provides optimal menus based on the user's personal information and also delivers meal kits and food. Specific embodiments of this system are described below.

[0823] First, users create an account using a smartphone or computer application, entering their email address and password. Once the account is created, a confirmation email is sent. Users then click the link in the email to verify their account.

[0824] Next, the user enters personal information such as family structure, age group, preferences, allergy information, and special dietary needs. This information is sent from the user's device to the server and stored in the database.

[0825] The server uses AI (artificial intelligence) to generate the optimal menu based on stored personal information. For example, if the family consists of two adults and one infant, their preference is for Japanese food, and they have peanut allergies, the server will generate a menu such as "Teriyaki Chicken," "Spinach with Sesame Dressing," and "Rice."

[0826] The generated menu is presented via the user's terminal. The user can select a special meal option (e.g., a menu for infants) from the presented menu. When the user selects a special meal, that information is sent to the server, which then prepares a meal kit corresponding to that special meal. For example, a meal kit is created that includes a menu with modified seasoning for infants.

[0827] Furthermore, users take photos of the ingredients in their refrigerator and upload the images to the application. The user's device sends the images to a server, which performs image analysis. For example, if the refrigerator contains cabbage, carrots, and pork, the server recognizes these ingredients and generates suggested menus such as "stir-fried pork and cabbage" and "carrot salad."

[0828] This ensures that fresh, personalized menus are always generated and presented to the user. These supplementary menus are presented through the user's device.

[0829] Users can order food directly based on the presented menu. The order information is sent to the server, and the logistics system delivers the food to the user's specified location (home or a refrigerated locker at the nearest supermarket).

[0830] To implement this system, software libraries such as "Python," "Requests," and "PIL" are used, along with devices such as smartphones and tablets.

[0831] As a concrete example, suppose a user inputs, "Please suggest a menu using the pork, cabbage, and carrots I have in my refrigerator." In response, the AI ​​uses the following prompt:

[0832] Prompt message:

[0833] I have pork, cabbage, and carrots in the refrigerator. My family consists of two adults and one infant. They have a peanut allergy. Please suggest a menu they can enjoy.

[0834] In this way, the system of the present invention provides users with personalized menu suggestions and meal kit delivery services, thereby reducing the burden of daily meal preparation.

[0835] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0836] Step 1:

[0837] Users create an account using a smartphone or computer application, entering their email address and password. The entered information is sent from the device to the server. The server stores the user's account information in a database and generates and sends a confirmation email to the user. The user authenticates their account by clicking a link in the confirmation email. In this step, the input is the email address and password, and the output is the authenticated user account.

[0838] Step 2:

[0839] Users enter personal information into the application, such as family structure, age group, preferences, allergy information, and special dietary needs. This information is sent from the terminal to the server and stored in a database. In this step, the input is personal information, and the output is the stored personal information.

[0840] Step 3:

[0841] The server uses AI (artificial intelligence) to generate the optimal menu based on stored personal information. The AI ​​generates prompt messages that take into account the user's family structure and preferences. For example, it might generate a prompt message such as, "Please suggest the optimal menu for a user with a family structure of 2 adults and 1 infant, a preference for Japanese food, and peanut allergies," and input this into the AI ​​model. The output is an optimized menu. In this step, the input is personal information, and the output is an optimized menu.

[0842] Step 4:

[0843] The server sends the generated optimized menu to the user's terminal. The terminal displays the suggested menu to the user. The user can select special meals, etc., from the presented menu. In this step, the input is the optimized menu, and the output is the menu displayed on the user's terminal.

[0844] Step 5:

[0845] The user takes a picture of the food in their refrigerator with their camera and uploads the image to the application. The image is sent from the device to the server. The server uses image analysis technology to recognize the food in the image. In this step, the input is an image of the food in the refrigerator, and the output is data of the recognized food.

[0846] Step 6:

[0847] The server generates a suggested menu based on the recognized ingredient data. For example, it generates a prompt such as, "There is pork, cabbage, and carrots in the refrigerator. Please suggest a menu using these ingredients," and inputs it into the AI ​​model. The output is a suggested menu. In this step, the input is the recognized ingredient data, and the output is a suggested menu.

[0848] Step 7:

[0849] The server sends the generated supplementary menu to the user's terminal. The terminal displays the suggested supplementary menu to the user. In this step, the input is the supplementary menu, and the output is the supplementary menu displayed on the user's terminal.

[0850] Step 8:

[0851] Users can order dishes based on the presented menu and supplementary menus. Order information is sent from the terminal to the server, which prepares the dishes via a logistics system and delivers them to the user's specified location. In this step, the input is the order information, and the output is the dishes delivered to the user's specified location.

[0852] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0853] This invention is a system that provides an optimal menu considering not only the user's family structure, preferences, and health condition, but also the user's emotions, and even delivers the meal kit. Specific embodiments of this system are described below.

[0854] First, the user creates an account. The user uses their device to enter their email address and password to create the account. Afterward, the user receives a confirmation email and clicks the link to authenticate their account. Next, they enter personal information such as family structure, age group, preferences, allergy information, and special dietary needs. This information is sent from the user's device to the server and stored in the database.

[0855] Next, the server uses AI to generate the optimal menu based on the stored personal information. For example, if the family consists of two adults and one infant, and their preference is for Japanese food, a menu such as "Teriyaki Chicken," "Spinach with Sesame Dressing," and "Rice" will be generated. The server also takes into account the user's past menu evaluation data to present an even more individually optimized menu.

[0856] The generated menu is presented to the user via a terminal. The user can select a special meal (e.g., a menu for infants) from the presented menu. When the user selects a special meal, that information is sent to the server, which generates a meal kit corresponding to the special meal. For example, for a family with an infant, a meal kit is prepared that includes teriyaki chicken seasoned for infants and soft spinach.

[0857] Next, the user can choose a delivery method, such as "home delivery" or "refrigerated locker at the nearest supermarket." If the user selects "home delivery," that information is sent to the server, and instructions are sent to the logistics system. As a result, the meal kit is delivered to the user's home.

[0858] Furthermore, users can take photos of the ingredients in their refrigerator and upload the images to the system. The device sends the images to the server, which then performs image analysis. For example, if the refrigerator contains cabbage, carrots, and pork, the server recognizes these ingredients and generates menus such as "stir-fried pork and cabbage" and "carrot salad." The generated menus are then presented to the user via the device.

[0859] Furthermore, this system includes an emotion engine that recognizes the user's emotions. Users can communicate their current mood and feelings to the system using an emotion input interface. For example, if a user inputs an emotion such as "I'm feeling stressed," the server will adjust the menu based on that information. For instance, it can suggest menus using ingredients with relaxing effects or recommend the user's favorite comfort foods.

[0860] The emotion engine continuously collects user emotional data and uses it for long-term preference analysis. This allows it to learn what emotions users have felt and which menu items they have preferred in the past, enabling it to provide even more personalized suggestions.

[0861] For example, if a user enters their family structure (2 adults, 1 infant), preferences (likes Japanese food), and allergies (peanuts), the system will suggest "Teriyaki Chicken," "Sesame Spinach," and "Rice." Furthermore, if the user selects the "Infant Menu" and specifies "Home Delivery," the meal kit will be delivered to the user's home. Also, if the user enters that they are "feeling stressed," the system will suggest additional items such as a "Matcha-flavored dessert" that has a relaxing effect.

[0862] In this way, the system of the present invention provides personalized menu suggestions and meal kit delivery services tailored to the user's needs and emotions, thereby reducing the burden of daily meal preparation and improving the quality of life.

[0863] The following describes the processing flow.

[0864] Step 1:

[0865] User: Go to the account creation screen and enter your email address and password.

[0866] Terminal: Sends the entered information to the server.

[0867] Server: Registers the email address and password in the authentication system and sends a confirmation email to the user.

[0868] Step 2:

[0869] User: Receive a confirmation email and click the link to verify your account.

[0870] Terminal: Displays an authentication screen and presents a form for entering family composition, age range, preferences, allergy information, and special dietary needs.

[0871] User: Enter information such as family composition (2 adults, 1 infant), age range (30s), preferences (likes Japanese food), and allergies (peanuts) into the input form.

[0872] Terminal: Sends the entered information to the server.

[0873] Server: Stores the received personal information in the database.

[0874] Step 3:

[0875] Server: Retrieves user personal information from the database.

[0876] Server: An AI algorithm generates the optimal menu based on personal information (e.g., "Teriyaki Chicken," "Sesame-dressed Spinach," "Rice").

[0877] Server: It also takes into account the user's past menu evaluation data to generate individually optimized menus (e.g., "Miso Soup").

[0878] Terminal: Displays the suggested menu on the user's screen.

[0879] Step 4:

[0880] User: Select a special meal (e.g., "Infant Menu") from the suggested menus.

[0881] Terminal: Sends the selected option information to the server.

[0882] Server: Generates meal kits tailored to special dietary needs (e.g., "Teriyaki Chicken (seasoned for infants)", "Soft Spinach").

[0883] Step 5:

[0884] User: Select a delivery method and specify either "Home Delivery" or "Refrigerated Locker at the nearest supermarket."

[0885] Terminal: Sends the selected shipping method information to the server.

[0886] Server: Issues instructions to the logistics system and prepares for the meal kits to be delivered to the user's specified location.

[0887] Step 6:

[0888] User: Take a picture of the ingredients in your refrigerator (e.g., "cabbage, carrots, pork") and upload the image to the system.

[0889] Terminal: Sends images to the server.

[0890] Server: Performs image analysis and generates menus based on recognized ingredients (e.g., "Stir-fried pork and cabbage", "Carrot salad").

[0891] Terminal: Displays the newly generated menu on the user's screen.

[0892] Step 7:

[0893] User: Use the emotion input interface to enter your current mood and emotions into the system (e.g., "I'm feeling stressed").

[0894] Terminal: Sends emotional data to the server.

[0895] Server: The emotion engine analyzes emotional data and adjusts the menu based on emotions (e.g., a relaxing "matcha dessert").

[0896] Terminal: Displays the adjusted new menu on the user's screen.

[0897] (Example 2)

[0898] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0899] Conventional menu generation systems suggest menus based on users' personal information and preferences, but they are insufficient in generating personalized menus that take into account users' emotions and refrigerator inventory, and in properly managing and delivering them. Furthermore, they lack sufficient functionality to optimize menus over the long term by utilizing users' past menu evaluations and emotional data, making it difficult to continuously increase user satisfaction.

[0900] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0901] In this invention, the server includes input means for inputting the user's personal information, menu generation means for generating a suitable menu based on the personal information and the user's emotional data, presentation means for presenting the generated menu to the user, preparation means for preparing a meal kit corresponding to a special meal selected by the user, delivery means for delivering the meal kit to the user, image analysis means for analyzing images of ingredients in the refrigerator uploaded by the user, auxiliary menu generation means for generating a menu based on the ingredients recognized by the image analysis means, auxiliary presentation means for presenting the generated auxiliary menu to the user, and evaluation data utilization means for further optimizing the menu based on the user's past menu evaluation data and emotional data. This makes it possible to provide and deliver optimized and personalized menus, and meal kits based on them, taking into account the user's personal information, emotional data, and the inventory in the refrigerator.

[0902] - A "user" refers to an individual who uses this system to create an account and provides personal information and sentiment data.

[0903] "Personal information" refers to information about the user, such as family structure, age group, preferences, allergy information, and special dietary needs.

[0904] "Emotional data" refers to information that users input, indicating their current mood and emotions.

[0905] "Menu generation means" refers to a function that generates appropriate menus based on personal information and emotional data.

[0906] "Presentation means" refers to a function for notifying or displaying the generated menu to the user.

[0907] "Special meals" refer to meal menus tailored to specific needs selected by the user.

[0908] "Preparation means" refers to the function of preparing meal kits that accommodate special dietary needs.

[0909] "Delivery method" refers to the function for delivering prepared meal kits to users.

[0910] "Image analysis means" refers to a function that analyzes images of food items in a refrigerator uploaded by the user to recognize the food items.

[0911] "Auxiliary menu generation means" refers to a function for generating menus based on ingredients recognized by image analysis means.

[0912] "Auxiliary presentation means" refers to a function for notifying or displaying the generated auxiliary menu to the user.

[0913] "Means of using evaluation data" refers to functions that further optimize menus based on the user's past menu evaluation data and sentiment data.

[0914] A "meal kit" refers to a set of ingredients prepared based on a pre-designed menu.

[0915] "Food in the refrigerator" refers to the food stored in the user's refrigerator.

[0916] This invention is a system that provides an optimal menu considering the user's personal information and emotional data, and even delivers the meal kit. Specifically, the system operates in the following steps.

[0917] First, the user accesses the system's webpage or app using their device. The user enters their email address and password to create an account. The server receives the entered information, registers the new user information in the database, and then sends a confirmation email. The user clicks the link in the email to authenticate their account. This completes the account creation process.

[0918] Next, the user enters personal information from their device, such as family structure, preferences, allergy information, and special diet needs. This information is sent to the server and stored in the database. For example, if the family consists of two adults and one infant, preferences are for Japanese food, and allergies include peanuts, this data will be stored in the user's profile.

[0919] The server uses stored personal information to call up an AI model (such as OpenAI's GPT-4) to generate an optimal menu. For example, if a family consists of two adults and one infant and prefers Japanese food, the AI ​​will suggest a menu such as "teriyaki chicken," "spinach with sesame dressing," and "rice." The server saves the generated menu in a database and presents it to the user via the terminal.

[0920] Users can review the menu and select special meals (e.g., infant menus). The selected information is sent to the server, which then prepares a meal kit tailored to the special meal. For example, the server might prepare a meal kit of "Teriyaki Chicken" with the seasoning modified for infants and include "Soft Spinach."

[0921] Next, the user selects a delivery method using a terminal. Options include "home delivery" and "refrigerated locker at the nearest supermarket." If the user selects "home delivery," that information is sent to the server. The server then instructs the logistics system to deliver the meal kit to the location specified by the user.

[0922] Furthermore, the user takes pictures of the food in the refrigerator with their device's camera and uploads the images to the server. The server uses an image analysis engine (such as the Google Cloud Vision API) to recognize the food. Based on the recognized food, the server uses an auxiliary menu generation method to generate the optimal menu and presents it to the user. For example, if the refrigerator contains cabbage, carrots, and pork, menus such as "stir-fried pork and cabbage" and "carrot salad" will be suggested.

[0923] Users input their current mood and emotions through their device. For example, they might input "I'm feeling stressed" or "I'm tired." The server adjusts the menu based on the emotional input data. For instance, if the user is feeling stressed, a relaxing dessert such as a "matcha-flavored dessert" might be suggested.

[0924] The server continuously collects user emotional data and provides even more personalized suggestions based on long-term preference analysis. For example, if a user previously preferred a particular menu item when they were feeling stressed, this information is used to optimize future suggestions.

[0925] Specific example:

[0926] The user enters their family composition (2 adults, 1 infant), preferences (Japanese food), and allergies (peanuts) into the terminal. The server receives this information, saves it to a database, and then calls an AI model to generate menus such as "Teriyaki Chicken," "Spinach with Sesame Dressing," and "Rice." The user reviews the suggestions on the terminal and selects a special meal (for the infant). Based on this information, the server prepares a meal kit, and if the user selects "Home Delivery," the meal kit is delivered. Additionally, if the user enters "I'm feeling stressed," the server will suggest additional menu items such as a relaxing "Matcha-flavored dessert."

[0927] Example of a prompt:

[0928] The user has entered their family structure (2 adults, 1 infant) and preferred Japanese food. Peanut allergy information has also been provided. Please suggest the most suitable menu and delivery options.

[0929] Thus, the system of the present invention provides personalized menu suggestions and meal kit delivery services tailored to the user's needs and emotions, thereby reducing the burden of daily meal preparation and improving the quality of life.

[0930] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0931] Step 1:

[0932] User account creation

[0933] The user accesses the system's webpage or app using their device. The user enters their email address and password and submits the account creation form. The server receives this information and registers the new user information in its database. The server sends a confirmation email to the registered email address. The user clicks the link in the email to authenticate their account, completing the account creation process.

[0934] Input: User's email address, password

[0935] Output: Account creation confirmation email

[0936] Step 2:

[0937] Input and storage of user personal information

[0938] Users use their devices to input personal information such as family structure, preferences, allergy information, and special dietary needs. The entered information is sent to the server, which stores it in a database. For example, if a user inputs that their family consists of two adults and one infant, their preferences are for Japanese food, and their allergy is peanuts, that information will be saved in the user's profile.

[0939] Input: Family composition, preferences, allergy information, special diet needs

[0940] Output: Personal information stored in the database

[0941] Step 3:

[0942] AI-powered optimal menu generation

[0943] The server uses stored personal information to call the appropriate AI model (e.g., a generative AI model). The AI ​​generates an optimal menu based on the personal information and emotional data. For example, if the family consists of two adults and one infant, and their preference is for Japanese food, the AI ​​will suggest a menu such as "teriyaki chicken," "spinach with sesame dressing," and "rice." The server saves the generated menu in a database.

[0944] Input: Personal Information

[0945] Output: Optimal menu

[0946] Step 4:

[0947] User-selected menu

[0948] The user reviews the suggested menu through their device. If a special diet (e.g., infant menu) is selected, the selection information is sent to the server, which stores it in a database. The process of generating a meal kit based on the special diet information is then initiated.

[0949] Input: Suggested menu, selection of special meals

[0950] Output: Special diet information stored on the server

[0951] Step 5:

[0952] Production of meal kits based on special diets

[0953] The server generates optimized meal kits based on the user's selected special diet information. For example, it might prepare a meal kit for infants with modified seasonings for "teriyaki chicken" and include "soft spinach." This information is managed by the server and used to integrate with the logistics system.

[0954] Input: Special diet information

[0955] Output: Optimized meal kit information

[0956] Step 6:

[0957] User selection of delivery method and delivery of meal kits

[0958] The user selects a delivery method using a terminal. Options include "home delivery" and "refrigerated locker at the nearest supermarket." If the user selects "home delivery," that information is sent to the server. The server then instructs the logistics system, and the meal kit is delivered to the location specified by the user.

[0959] Input: Select shipping method

[0960] Output: Delivery Instructions

[0961] Step 7:

[0962] Image recognition of food items inside a refrigerator

[0963] The user takes a picture of the food in their refrigerator with their camera and uploads the image from their device to the server. The server uses an image analysis engine (e.g., an image analysis API) to recognize the food in the image. Based on the recognized food, the server generates an optimal menu and presents it to the user.

[0964] Input: Image of food items in the refrigerator

[0965] Output: Optimal menu

[0966] Step 8:

[0967] User emotional input and menu adjustment

[0968] Users input their current mood and emotions through their device, such as "I'm stressed" or "I'm tired." The device sends this information to a server. The server adjusts the menu based on the emotional input data. For example, if the user is feeling stressed, a relaxing dessert such as a "matcha-flavored dessert" might be suggested.

[0969] Input: Current mood or feelings

[0970] Output: Adjusted menu

[0971] Step 9:

[0972] Accumulation of emotional data and long-term preference analysis

[0973] The server continuously accumulates user emotional data. This stored emotional data is used for long-term preference analysis. This allows the system to learn what emotions users have experienced and which menu items they have preferred in the past, enabling more personalized suggestions. For example, if a user has preferred a particular menu item when feeling stressed in the past, this information can be used to optimize future suggestions.

[0974] Input: Sentiment data

[0975] Output: Personalized suggestions

[0976] (Application Example 2)

[0977] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0978] In modern dietary habits, it is difficult for users to create personalized menus tailored to their family structure, preferences, health status, and even their emotions, and then quickly prepare meal kits based on those menus. Furthermore, there is no consistent system for efficiently utilizing ingredients in the refrigerator while providing meals that meet individual needs. As a result, users often experience stress and inconvenience.

[0979] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes an input means for inputting the user's personal information, a menu generation means for generating a suitable menu based on the personal information, a presentation means for presenting the generated menu to the user, a preparation means for preparing a meal kit corresponding to the selected special diet, a delivery means for delivering the meal kit to the user, an emotion input means for inputting or recognizing the user's emotions, and an emotion adjustment means for adjusting the menu based on the emotion input means. This enables personalized menu suggestions that respond to the user's individual needs and emotions, as well as efficient delivery of meal kits.

[0980] "User personal information" refers to information related to an individual user, such as family structure, age group, preferences, allergy information, and special dietary needs.

[0981] An "input method" refers to an interface or device that allows a user to input personal information or emotional data into a system.

[0982] A "menu generation method" is a means for generating an optimal menu based on the user's personal information and emotional data.

[0983] "Presentation means" refers to means of presenting the generated menu to the user visually or audibly.

[0984] "Preparation methods" refer to the means of preparing meal kits that correspond to the selected special diet.

[0985] "Delivery method" refers to the means by which prepared meal kits are delivered to users.

[0986] An "emotion input method" is a means of inputting or recognizing a user's emotions.

[0987] "Emotion adjustment methods" refer to methods for adjusting menus based on the user's emotional data.

[0988] "Image analysis means" refers to a means for analyzing images of food items inside a refrigerator that have been uploaded by a user.

[0989] The "supplementary menu generation means" is a means for generating supplementary menus based on ingredients recognized by the image analysis means.

[0990] "Auxiliary presentation means" refers to a means for presenting menus generated by the auxiliary menu generation means to the user.

[0991] This invention is a system that generates menus and provides / delivers meal kits, taking into account the user's family structure, preferences, health status, and emotions. The system operates when the user creates an account via smartphone, enters personal information, uploads images of ingredients in their refrigerator, or enters their emotions.

[0992] First, users create an account using their smartphone. They enter their email address and password to create the account. Next, they enter personal information such as family structure, age group, preferences, allergy information, and special dietary needs. This information is sent from the device to the server and stored in the database.

[0993] After personal information is saved, the server uses AI to generate an optimal menu based on the saved information. For example, if the family consists of two adults and one infant, and their preference is for Japanese food, a menu such as "Teriyaki Chicken," "Spinach with Sesame Dressing," and "Rice" will be generated. Furthermore, the menu is adjusted using an emotion input method that allows the user to input or recognize their emotions. If the user inputs that they are feeling stressed, the menu will suggest dishes using ingredients with relaxing effects or their favorite comfort foods.

[0994] The generated menu is presented to the user via a terminal. The user can select a special meal (for example, a menu for infants) from the presented menu. When the user selects a special meal, that information is sent to the server, which generates a meal kit corresponding to the special meal. For example, for a family with an infant, a meal kit is prepared that includes teriyaki chicken seasoned for infants and soft spinach. Next, the user selects a delivery method and chooses "home delivery," at which point the meal kit is delivered to the user's home.

[0995] Furthermore, users can take photos of the ingredients in their refrigerator and upload the images to the system. The device sends the images to the server, which then performs image analysis. For example, if the refrigerator contains cabbage, carrots, and pork, the server recognizes these ingredients and generates menus such as "stir-fried pork and cabbage" and "carrot salad." The generated menus are then presented to the user via the device.

[0996] The hardware and software used include user terminals (e.g., smartphones), image recognition technology (e.g., OpenCV, TensorFlow), and emotion recognition technology (e.g., EmotionRecognition model). Menus are generated using AI, and deep learning models (e.g., TensorFlow or PyTorch) are used on the server side.

[0997] For example, if a user enters their family structure (2 adults, 1 infant), preferences (likes Japanese food), and allergies (peanuts), the system will suggest "Teriyaki Chicken," "Sesame Spinach," and "Rice." Furthermore, if the user selects the "Infant Menu" and specifies "Home Delivery," the meal kit will be delivered to the user's home. Also, if the user enters that they are "feeling stressed," the system will suggest additional items such as a "Matcha-flavored dessert" that has a relaxing effect.

[0998] Examples of specific prompt statements include the following:

[0999] "Based on the following profile and emotional data, please suggest the following menu: User Profile: Family Composition: 2 adults, 1 infant; Preferences: Japanese food; Allergies: Peanuts; Special Diet Needs: Infant menu; Emotions: Feeling stressed."

[1000] By fulfilling these requirements, the system can provide personalized menu suggestions and meal kit delivery services tailored to the user's needs and emotions.

[1001] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1002] Step 1:

[1003] The user creates an account using their smartphone. The input consists of an email address and password, which the device receives and sends to the server. The server receives this information, registers the user account in its database, and sends a confirmation email. The output is a confirmation email indicating successful account creation.

[1004] Step 2:

[1005] The user enters personal information such as family structure, age group, preferences, allergy information, and special dietary needs. This information is sent from the terminal to the server, which stores this data in a database. The output confirms the saving of the personal information.

[1006] Step 3:

[1007] The server uses AI to generate the optimal menu based on stored personal information. In this process, personal information stored in a database is used as input, and a generation AI model is used to calculate and generate the menu. The output is the generated menu data.

[1008] Step 4:

[1009] The terminal displays the generated menu to the user. The input is menu data received from the server, which the terminal displays to the user. The output is the menu presented to the user.

[1010] Step 5:

[1011] The user selects a special meal from the presented menu. The user's selection is sent from the terminal to the server, which verifies this information. The input is the user's selected special meal information, and the server prepares the meal kit based on it. The output is the data for the prepared meal kit.

[1012] Step 6:

[1013] The user selects a shipping method. The user's selected shipping method is sent from the terminal to the server, which then issues instructions to the logistics system. The input is the user's selected shipping method, and the output is a notification that the shipping arrangements have been completed.

[1014] Step 7:

[1015] The user takes a picture of the food in their refrigerator and uploads the image from their device to the server. The server uses image analysis to recognize the food. The input is an image of the food in the refrigerator, and the output is the analyzed food data.

[1016] Step 8:

[1017] The server generates supplementary menus based on recognized ingredient data. In this process, the analyzed ingredient data is used as input, and a generation AI model is used to calculate and generate the supplementary menus. The output is the data of the generated supplementary menus.

[1018] Step 9:

[1019] The terminal displays the generated supplementary menu to the user. The input is the supplementary menu data received from the server, which the terminal displays to the user. The output is the supplementary menu presented to the user.

[1020] Step 10:

[1021] The user inputs an emotion or uploads a facial image. The device sends this data to the server, which uses the emotion input method to recognize the emotion. The input is the user's emotion or facial image data, and the output is the recognized emotion data.

[1022] Step 11:

[1023] The server adjusts the menu based on recognized emotion data. This process uses emotion data as input and readjusts the menu using emotion adjustment mechanisms. The output is the adjusted menu data.

[1024] This processing flow enables the delivery of meal kits and menu suggestions optimized to the individual needs and emotions of each user.

[1025] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1026] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1027] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1028] [Fourth Embodiment]

[1029] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1030] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1031] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1032] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1033] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1034] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1035] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1036] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1037] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1038] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1039] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1040] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1041] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1042] This invention is a system that provides an optimal menu based on the user's personal information, such as family structure, preferences, and health status, and even delivers the meal kit. Specific embodiments of this system are described below.

[1043] First, the user creates an account. The user uses their device to enter their email address and password to create the account. Afterward, the user receives a confirmation email and clicks the link to authenticate their account. Next, they enter personal information such as family structure, age group, preferences, allergy information, and special dietary needs. This information is sent from the user's device to the server and stored in the database.

[1044] Next, the server uses AI to generate the optimal menu based on the stored personal information. For example, if the family consists of two adults and one infant, and their preference is for Japanese food, a menu such as "Teriyaki Chicken," "Spinach with Sesame Dressing," and "Rice" will be generated. The server also takes into account the user's past menu evaluation data to present an even more individually optimized menu.

[1045] The generated menu is presented to the user via a terminal. The user can select a special meal (e.g., a menu for infants) from the presented menu. When the user selects a special meal, that information is sent to the server, which generates a meal kit corresponding to the special meal. For example, for a family with an infant, a meal kit is prepared that includes teriyaki chicken seasoned for infants and soft spinach.

[1046] Next, the user can choose a delivery method, such as "home delivery" or "refrigerated locker at the nearest supermarket." If the user selects "home delivery," that information is sent to the server, and instructions are sent to the logistics system. As a result, the meal kit is delivered to the user's home.

[1047] Furthermore, users can take photos of the ingredients in their refrigerator and upload the images to the system. The device sends the images to the server, which then performs image analysis. For example, if the refrigerator contains cabbage, carrots, and pork, the server recognizes these ingredients and generates menus such as "stir-fried pork and cabbage" and "carrot salad." The generated menus are then presented to the user via the device.

[1048] As a concrete example, the following process can be considered: When the user enters their family composition (2 adults, 1 infant), preferences (likes Japanese food), and allergies (peanuts), the system suggests "Teriyaki Chicken," "Sesame Spinach," and "Rice" based on that information. Furthermore, if the user selects the "Infant Menu" and specifies "Home Delivery," the meal kit will be delivered to the user's home.

[1049] In this way, the system of the present invention can provide personalized menu suggestions and meal kit delivery services tailored to the user's needs, thereby reducing the burden of daily meal preparation.

[1050] The following describes the processing flow.

[1051] Step 1:

[1052] User: Go to the account creation screen and enter your email address and password.

[1053] Terminal: Sends the entered information to the server.

[1054] Server: Registers the email address and password in the authentication system and sends a confirmation email to the user.

[1055] Step 2:

[1056] User: Receive a confirmation email and click the link to verify your account.

[1057] Terminal: Displays an authentication screen and presents a form for entering family composition, age range, preferences, allergy information, and special dietary needs.

[1058] User: Enter information such as family composition (2 adults, 1 infant), age range (30s), preferences (likes Japanese food), and allergies (peanuts) into the input form.

[1059] Terminal: Sends the entered information to the server.

[1060] Server: Stores the received personal information in the database.

[1061] Step 3:

[1062] Server: Retrieves user personal information from the database.

[1063] Server: An AI algorithm generates the optimal menu based on personal information (e.g., "Teriyaki Chicken," "Sesame-dressed Spinach," "Rice").

[1064] Server: It also takes into account the user's past menu evaluation data to generate individually optimized menus (e.g., "Miso Soup").

[1065] Terminal: Displays the suggested menu on the user's screen.

[1066] Step 4:

[1067] User: Select a special meal (e.g., "Infant Menu") from the suggested menus.

[1068] Terminal: Sends the selected option information to the server.

[1069] Server: Generates meal kits tailored to special dietary needs (e.g., "Teriyaki Chicken (seasoned for infants)", "Soft Spinach").

[1070] Step 5:

[1071] User: Select a delivery method and specify either "Home Delivery" or "Refrigerated Locker at the nearest supermarket."

[1072] Terminal: Sends the selected shipping method information to the server.

[1073] Server: Issues instructions to the logistics system and prepares for the meal kits to be delivered to the user's specified location.

[1074] Step 6:

[1075] User: Take a picture of the ingredients in your refrigerator (e.g., "cabbage, carrots, pork") and upload the image to the system.

[1076] Terminal: Sends images to the server.

[1077] Server: Performs image analysis and generates menus based on recognized ingredients (e.g., "Stir-fried pork and cabbage", "Carrot salad").

[1078] Terminal: Displays the newly generated menu on the user's screen.

[1079] (Example 1)

[1080] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1081] Conventional meal planning systems struggle to propose individually optimized menus based on users' personal information, and their suggestions tend to be biased. Furthermore, they often lack effective ways to utilize the ingredients users already possess, leading to food waste. To address these issues, there is a need for a system that generates optimal meal plans tailored to each user's individual circumstances and delivers meal kits based on those plans.

[1082] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1083] In this invention, the server includes input means for inputting the user's personal information, meal plan generation means for generating an optimal meal plan based on the personal information, presentation means for presenting the generated meal plan to the user, preparation means for preparing an ingredient kit corresponding to the selected specific food, and delivery means for delivering the ingredient kit to the user. This enables the generation of a personalized meal plan tailored to the user's individual circumstances and the delivery of an ingredient kit based on that meal plan.

[1084] "User personal information" refers to information such as the user's family structure, age group, preferences, allergy information, and special dietary needs.

[1085] "Input method" refers to an interface for users to enter personal information, including input forms and applications on a device.

[1086] "Meal plan generation method" refers to algorithms or AI models that generate optimal meal plans based on the user's personal information.

[1087] "Presentation means" refers to a method for displaying the generated meal plan to the user, and includes the terminal's display and notification system.

[1088] "Specific foods" refer to ingredients or menu items selected by users based on their specific needs or requests.

[1089] "Preparation means" refers to a system for preparing ingredient kits corresponding to selected specific foods, including the selection and packing of ingredients.

[1090] "Delivery method" refers to the means of delivering prepared food kits to a location specified by the user, and includes logistics systems and delivery services.

[1091] "Images of food in storage" refers to images taken by users of food in their home refrigerators or food storage areas.

[1092] "Image analysis means" refers to a technology for analyzing food images uploaded by users and recognizing the ingredients contained within them, and includes an image recognition algorithm.

[1093] "Supplementary meal plan generation means" refers to algorithms or AI models for generating supplementary meal plans based on ingredients recognized by image analysis means.

[1094] "Auxiliary presentation means" refers to a method for displaying a meal plan generated by the auxiliary meal plan generation means to the user, and includes the terminal's display and notification system.

[1095] "Methods for using evaluation data" refers to systems and algorithms that utilize evaluation data from users' past meal plans to further optimize meal plans.

[1096] This invention is a system that provides an optimal meal plan based on the user's personal information and even delivers the ingredient kit. This system consists of three components: a server, a terminal, and the user. Specific embodiments of this system are described below.

[1097] First, the user creates an account using their device. The user enters their email address and password and registers on the account creation page. This information is sent from the device to the server, which stores it in its database. The server then sends a confirmation email to the user, and the user clicks the confirmation link in the email to authenticate their account.

[1098] Next, the user logs into their account and enters personal information such as family structure, age range, preferences, and allergy information. The entered information is sent from the device to the server and stored in the database.

[1099] The server uses an AI model to generate an optimal meal plan based on stored personal information. For example, if the family consists of two adults and one infant, prefers Japanese food, and has a peanut allergy, a menu such as "Teriyaki Chicken," "Spinach with Sesame Dressing," and "Rice" will be suggested. Because this process uses a generative AI model, the following input is used as a prompt.

[1100] Example of a prompt:

[1101] "The user's family consists of two adults and one infant, and they prefer Japanese food. They also have a peanut allergy. Please suggest the best menu based on these conditions."

[1102] The generated meal plan is sent from the server to the user's terminal, where the user can view the suggested menu on their terminal screen. The user can select specific foods from this menu (for example, infant menus), and this selection information is sent from the terminal to the server. Based on this information, the server prepares ingredient kits corresponding to the selected foods.

[1103] Furthermore, users can choose their delivery method. For example, they might have options like "home delivery" or "refrigerated locker at the nearest supermarket." The selected delivery method is then sent from the terminal to the server. The server sends instructions to the logistics system, and the meal kit is delivered to the user using the specified delivery method.

[1104] To utilize the ingredients in their storage, users take photos of the food in their refrigerator and upload the images to their device. The device sends these images to a server, which performs image analysis to recognize the ingredients. For example, if the refrigerator contains cabbage, carrots, and pork, the server will suggest supplementary menus based on these ingredients, such as "stir-fried pork and cabbage" or "carrot salad." This information is also sent from the server to the user's device.

[1105] This system supports users in their daily meal preparation and contributes to reducing food waste by generating an optimal meal plan tailored to each user's individual needs and providing delivery of ingredient kits based on that plan.

[1106] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1107] Step 1:

[1108] The user enters their email address and password on the device's registration screen and presses the "Create Account" button. This information is sent from the device to the server. The input includes an email address (e.g., example@example.com) and a password. An account creation request is sent to the server as output.

[1109] Step 2:

[1110] The server stores the received account information in a database and sends a confirmation email to the user's email address. As part of data processing, the email address and password are hashed and stored. The confirmation email is then sent to the user as output.

[1111] Example: The server stores the email address in a database and sends a confirmation email using the SMTP protocol.

[1112] Step 3:

[1113] The user clicks the link in the confirmation email to authenticate their account. This action sends the user's request back to the server, updating the account status to "Authenticated." The input is the click of the email link, and the output is the updated account authentication status.

[1114] Step 4:

[1115] After logging in, users enter personal information such as family structure, age group, preferences, and allergy information. This information is sent from the device to the server and stored in the database. Personal information is included as input, and the user profile is updated as output.

[1116] Example: The user enters information such as "2 adults, 1 infant," "likes Japanese food," and "peanut allergy."

[1117] Step 5:

[1118] The server uses an AI model to generate an optimal meal plan based on stored personal information. As part of the data processing, personal information is converted into prompt sentences and input into the AI ​​model. The generated meal plan is obtained as output.

[1119] Example prompt: "The user's family consists of two adults and one infant, and they prefer Japanese food. They have a peanut allergy. Please suggest the best menu based on these conditions."

[1120] Step 6:

[1121] The server sends the generated meal plan to the user's terminal. As output, the meal plan data is sent to the terminal.

[1122] Example: The server sends data in JSON format and converts it for display on the terminal.

[1123] Step 7:

[1124] The user reviews the suggested meal plan via their device and selects specific foods (e.g., infant menu). This selection information is sent from the device to the server as input, and then saved to the server as output.

[1125] Step 8:

[1126] The server generates information to prepare an ingredient kit corresponding to the selected specific food item. The kit contents are determined based on the specific food item information as part of the data processing. Preparation instructions are then generated as output.

[1127] Example: Modify the seasoning of "Teriyaki Chicken" to suit infants and prepare a meal kit that includes soft spinach.

[1128] Step 9:

[1129] The user selects a shipping method and sends the information from their device to the server. The input includes the selected shipping method, and the output is stored on the server.

[1130] Step 10:

[1131] The server issues instructions to the logistics system, which then delivers the food kits to the user using the specified delivery method. The data calculation generates delivery instructions for the logistics system, and the output is the execution of those instructions.

[1132] Example: A meal kit is delivered to the user's home.

[1133] Step 11:

[1134] The user takes a picture of the food in the refrigerator and uploads the image to their device. The input includes the captured image data, and the output is the image data sent to the server.

[1135] Step 12:

[1136] The server performs image analysis to recognize ingredients from the input image. As part of data processing, it analyzes the image data and extracts ingredient information. The output is a list of the recognized ingredients.

[1137] Example: Image analysis identifies cabbage, carrots, and pork.

[1138] Step 13:

[1139] The server generates a supplementary meal plan based on the recognized ingredients and sends it to the user's terminal. As part of data processing, the recognized ingredient information is input into an AI model to generate a supplementary meal plan. As output, the supplementary meal plan is generated and sent to the terminal.

[1140] Examples: "Stir-fried pork and cabbage" and "Carrot salad" are suggested.

[1141] In this way, by adding specific actions at each processing step and clearly indicating data input and output, personalized meal planning and ingredient kit delivery tailored to the user's needs can be realized.

[1142] (Application Example 1)

[1143] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1144] While conventional systems offered menu suggestions and meal kit delivery based on users' personal information, they lacked the functionality to effectively utilize ingredients in the refrigerator and propose more precisely personalized menus. Furthermore, the lack of integrated food delivery services that allowed users to directly order meals based on the suggested menus meant that user convenience was not sufficiently improved. Therefore, there is a need for a system that enables food suggestions and delivery based on individual user needs and real-time circumstances.

[1145] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1146] In this invention, the server includes input means for inputting the user's personal information, menu generation means for generating a suitable menu based on the personal information, presentation means for presenting the generated menu to the user, preparation means for preparing a meal kit corresponding to the selected special diet, delivery means for delivering the meal kit to the user, image analysis means for uploading images of ingredients in the refrigerator and analyzing those images, auxiliary menu generation means for generating auxiliary menus based on the ingredients recognized by the image analysis means, auxiliary presentation means for presenting the auxiliary menus generated by the auxiliary menu generation means to the user, and ordering means for ordering dishes based on the provided menu. As a result, the user can have an optimal menu suggested based on their personal information and the ingredients in their refrigerator, and can also directly order dishes according to that menu.

[1147] "User personal information" refers to individual information such as the user's family structure, age group, preferences, allergy information, and special dietary needs.

[1148] An "input method" is an interface for users to input personal information, and may consist of forms provided on the screen of a smartphone or computer, for example.

[1149] A "menu generation method" is a system that uses artificial intelligence (AI) to generate the optimal menu based on the user's personal information.

[1150] "Presentation means" refers to an interface for displaying the generated menu to the user, and includes the display of a smartphone or tablet.

[1151] "Preparation methods" refer to the means of preparing meal kits that correspond to the special diet selected by the user.

[1152] "Delivery method" refers to the means by which meal kits are delivered to users, including logistics systems and delivery companies.

[1153] "Image analysis means" refers to a technology for analyzing images of food items inside a refrigerator and recognizing them, and is a system that utilizes computer vision and machine learning algorithms.

[1154] The "supplementary menu generation means" is a means for generating additional menus based on ingredients recognized by the image analysis means.

[1155] The "auxiliary presentation means" is an interface for presenting menus generated by the auxiliary menu generation means to the user.

[1156] "Ordering method" refers to a means of ordering food based on the provided menu, and includes, for example, the ordering screen of a smartphone application.

[1157] This invention is a system that provides optimal menus based on the user's personal information and also delivers meal kits and food. Specific embodiments of this system are described below.

[1158] First, users create an account using a smartphone or computer application, entering their email address and password. Once the account is created, a confirmation email is sent. Users then click the link in the email to verify their account.

[1159] Next, the user enters personal information such as family structure, age group, preferences, allergy information, and special dietary needs. This information is sent from the user's device to the server and stored in the database.

[1160] The server uses AI (artificial intelligence) to generate the optimal menu based on stored personal information. For example, if the family consists of two adults and one infant, their preference is for Japanese food, and they have peanut allergies, the server will generate a menu such as "Teriyaki Chicken," "Spinach with Sesame Dressing," and "Rice."

[1161] The generated menu is presented via the user's terminal. The user can select a special meal option (e.g., a menu for infants) from the presented menu. When the user selects a special meal, that information is sent to the server, which then prepares a meal kit corresponding to that special meal. For example, a meal kit is created that includes a menu with modified seasoning for infants.

[1162] Furthermore, users take photos of the ingredients in their refrigerator and upload the images to the application. The user's device sends the images to a server, which performs image analysis. For example, if the refrigerator contains cabbage, carrots, and pork, the server recognizes these ingredients and generates suggested menus such as "stir-fried pork and cabbage" and "carrot salad."

[1163] This ensures that fresh, personalized menus are always generated and presented to the user. These supplementary menus are presented through the user's device.

[1164] Users can order food directly based on the presented menu. The order information is sent to the server, and the logistics system delivers the food to the user's specified location (home or a refrigerated locker at the nearest supermarket).

[1165] To implement this system, software libraries such as "Python," "Requests," and "PIL" are used, along with devices such as smartphones and tablets.

[1166] As a concrete example, suppose a user inputs, "Please suggest a menu using the pork, cabbage, and carrots I have in my refrigerator." In response, the AI ​​uses the following prompt:

[1167] Prompt message:

[1168] I have pork, cabbage, and carrots in the refrigerator. My family consists of two adults and one infant. They have a peanut allergy. Please suggest a menu they can enjoy.

[1169] In this way, the system of the present invention provides users with personalized menu suggestions and meal kit delivery services, thereby reducing the burden of daily meal preparation.

[1170] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1171] Step 1:

[1172] Users create an account using a smartphone or computer application, entering their email address and password. The entered information is sent from the device to the server. The server stores the user's account information in a database and generates and sends a confirmation email to the user. The user authenticates their account by clicking a link in the confirmation email. In this step, the input is the email address and password, and the output is the authenticated user account.

[1173] Step 2:

[1174] Users enter personal information into the application, such as family structure, age group, preferences, allergy information, and special dietary needs. This information is sent from the terminal to the server and stored in a database. In this step, the input is personal information, and the output is the stored personal information.

[1175] Step 3:

[1176] The server uses AI (artificial intelligence) to generate the optimal menu based on stored personal information. The AI ​​generates prompt messages that take into account the user's family structure and preferences. For example, it might generate a prompt message such as, "Please suggest the optimal menu for a user with a family structure of 2 adults and 1 infant, a preference for Japanese food, and peanut allergies," and input this into the AI ​​model. The output is an optimized menu. In this step, the input is personal information, and the output is an optimized menu.

[1177] Step 4:

[1178] The server sends the generated optimized menu to the user's terminal. The terminal displays the suggested menu to the user. The user can select special meals, etc., from the presented menu. In this step, the input is the optimized menu, and the output is the menu displayed on the user's terminal.

[1179] Step 5:

[1180] The user takes a picture of the food in their refrigerator with their camera and uploads the image to the application. The image is sent from the device to the server. The server uses image analysis technology to recognize the food in the image. In this step, the input is an image of the food in the refrigerator, and the output is data of the recognized food.

[1181] Step 6:

[1182] The server generates a suggested menu based on the recognized ingredient data. For example, it generates a prompt such as, "There is pork, cabbage, and carrots in the refrigerator. Please suggest a menu using these ingredients," and inputs it into the AI ​​model. The output is a suggested menu. In this step, the input is the recognized ingredient data, and the output is a suggested menu.

[1183] Step 7:

[1184] The server sends the generated supplementary menu to the user's terminal. The terminal displays the suggested supplementary menu to the user. In this step, the input is the supplementary menu, and the output is the supplementary menu displayed on the user's terminal.

[1185] Step 8:

[1186] Users can order dishes based on the presented menu and supplementary menus. Order information is sent from the terminal to the server, which prepares the dishes via a logistics system and delivers them to the user's specified location. In this step, the input is the order information, and the output is the dishes delivered to the user's specified location.

[1187] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1188] This invention is a system that provides an optimal menu considering not only the user's family structure, preferences, and health condition, but also the user's emotions, and even delivers the meal kit. Specific embodiments of this system are described below.

[1189] First, the user creates an account. The user uses their device to enter their email address and password to create the account. Afterward, the user receives a confirmation email and clicks the link to authenticate their account. Next, they enter personal information such as family structure, age group, preferences, allergy information, and special dietary needs. This information is sent from the user's device to the server and stored in the database.

[1190] Next, the server uses AI to generate the optimal menu based on the stored personal information. For example, if the family consists of two adults and one infant, and their preference is for Japanese food, a menu such as "Teriyaki Chicken," "Spinach with Sesame Dressing," and "Rice" will be generated. The server also takes into account the user's past menu evaluation data to present an even more individually optimized menu.

[1191] The generated menu is presented to the user via a terminal. The user can select a special meal (e.g., a menu for infants) from the presented menu. When the user selects a special meal, that information is sent to the server, which generates a meal kit corresponding to the special meal. For example, for a family with an infant, a meal kit is prepared that includes teriyaki chicken seasoned for infants and soft spinach.

[1192] Next, the user can choose a delivery method, such as "home delivery" or "refrigerated locker at the nearest supermarket." If the user selects "home delivery," that information is sent to the server, and instructions are sent to the logistics system. As a result, the meal kit is delivered to the user's home.

[1193] Furthermore, users can take photos of the ingredients in their refrigerator and upload the images to the system. The device sends the images to the server, which then performs image analysis. For example, if the refrigerator contains cabbage, carrots, and pork, the server recognizes these ingredients and generates menus such as "stir-fried pork and cabbage" and "carrot salad." The generated menus are then presented to the user via the device.

[1194] Furthermore, this system includes an emotion engine that recognizes the user's emotions. Users can communicate their current mood and feelings to the system using an emotion input interface. For example, if a user inputs an emotion such as "I'm feeling stressed," the server will adjust the menu based on that information. For instance, it can suggest menus using ingredients with relaxing effects or recommend the user's favorite comfort foods.

[1195] The emotion engine continuously collects user emotional data and uses it for long-term preference analysis. This allows it to learn what emotions users have felt and which menu items they have preferred in the past, enabling it to provide even more personalized suggestions.

[1196] For example, if a user enters their family structure (2 adults, 1 infant), preferences (likes Japanese food), and allergies (peanuts), the system will suggest "Teriyaki Chicken," "Sesame Spinach," and "Rice." Furthermore, if the user selects the "Infant Menu" and specifies "Home Delivery," the meal kit will be delivered to the user's home. Also, if the user enters that they are "feeling stressed," the system will suggest additional items such as a "Matcha-flavored dessert" that has a relaxing effect.

[1197] In this way, the system of the present invention provides personalized menu suggestions and meal kit delivery services tailored to the user's needs and emotions, thereby reducing the burden of daily meal preparation and improving the quality of life.

[1198] The following describes the processing flow.

[1199] Step 1:

[1200] User: Go to the account creation screen and enter your email address and password.

[1201] Terminal: Sends the entered information to the server.

[1202] Server: Registers the email address and password in the authentication system and sends a confirmation email to the user.

[1203] Step 2:

[1204] User: Receive a confirmation email and click the link to verify your account.

[1205] Terminal: Displays an authentication screen and presents a form for entering family composition, age range, preferences, allergy information, and special dietary needs.

[1206] User: Enter information such as family composition (2 adults, 1 infant), age range (30s), preferences (likes Japanese food), and allergies (peanuts) into the input form.

[1207] Terminal: Sends the entered information to the server.

[1208] Server: Stores the received personal information in the database.

[1209] Step 3:

[1210] Server: Retrieves user personal information from the database.

[1211] Server: An AI algorithm generates the optimal menu based on personal information (e.g., "Teriyaki Chicken," "Sesame-dressed Spinach," "Rice").

[1212] Server: It also takes into account the user's past menu evaluation data to generate individually optimized menus (e.g., "Miso Soup").

[1213] Terminal: Displays the suggested menu on the user's screen.

[1214] Step 4:

[1215] User: Select a special meal (e.g., "Infant Menu") from the suggested menus.

[1216] Terminal: Sends the selected option information to the server.

[1217] Server: Generates meal kits tailored to special dietary needs (e.g., "Teriyaki Chicken (seasoned for infants)", "Soft Spinach").

[1218] Step 5:

[1219] User: Select a delivery method and specify either "Home Delivery" or "Refrigerated Locker at the nearest supermarket."

[1220] Terminal: Sends the selected shipping method information to the server.

[1221] Server: Issues instructions to the logistics system and prepares for the meal kits to be delivered to the user's specified location.

[1222] Step 6:

[1223] User: Take a picture of the ingredients in your refrigerator (e.g., "cabbage, carrots, pork") and upload the image to the system.

[1224] Terminal: Sends images to the server.

[1225] Server: Performs image analysis and generates menus based on recognized ingredients (e.g., "Stir-fried pork and cabbage", "Carrot salad").

[1226] Terminal: Displays the newly generated menu on the user's screen.

[1227] Step 7:

[1228] User: Use the emotion input interface to enter your current mood and emotions into the system (e.g., "I'm feeling stressed").

[1229] Terminal: Sends emotional data to the server.

[1230] Server: The emotion engine analyzes emotional data and adjusts the menu based on emotions (e.g., a relaxing "matcha dessert").

[1231] Terminal: Displays the adjusted new menu on the user's screen.

[1232] (Example 2)

[1233] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1234] Conventional menu generation systems suggest menus based on users' personal information and preferences, but they are insufficient in generating personalized menus that take into account users' emotions and refrigerator inventory, and in properly managing and delivering them. Furthermore, they lack sufficient functionality to optimize menus over the long term by utilizing users' past menu evaluations and emotional data, making it difficult to continuously increase user satisfaction.

[1235] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1236] In this invention, the server includes input means for inputting the user's personal information, menu generation means for generating a suitable menu based on the personal information and the user's emotional data, presentation means for presenting the generated menu to the user, preparation means for preparing a meal kit corresponding to a special meal selected by the user, delivery means for delivering the meal kit to the user, image analysis means for analyzing images of ingredients in the refrigerator uploaded by the user, auxiliary menu generation means for generating a menu based on the ingredients recognized by the image analysis means, auxiliary presentation means for presenting the generated auxiliary menu to the user, and evaluation data utilization means for further optimizing the menu based on the user's past menu evaluation data and emotional data. This makes it possible to provide and deliver optimized and personalized menus, and meal kits based on them, taking into account the user's personal information, emotional data, and the inventory in the refrigerator.

[1237] - A "user" refers to an individual who uses this system to create an account and provides personal information and sentiment data.

[1238] "Personal information" refers to information about the user, such as family structure, age group, preferences, allergy information, and special dietary needs.

[1239] "Emotional data" refers to information that users input, indicating their current mood and emotions.

[1240] "Menu generation means" refers to a function that generates appropriate menus based on personal information and emotional data.

[1241] "Presentation means" refers to a function for notifying or displaying the generated menu to the user.

[1242] "Special meals" refer to meal menus tailored to specific needs selected by the user.

[1243] "Preparation means" refers to the function of preparing meal kits that accommodate special dietary needs.

[1244] "Delivery method" refers to the function for delivering prepared meal kits to users.

[1245] "Image analysis means" refers to a function that analyzes images of food items in a refrigerator uploaded by the user to recognize the food items.

[1246] "Auxiliary menu generation means" refers to a function for generating menus based on ingredients recognized by image analysis means.

[1247] "Auxiliary presentation means" refers to a function for notifying or displaying the generated auxiliary menu to the user.

[1248] "Means of using evaluation data" refers to functions that further optimize menus based on the user's past menu evaluation data and sentiment data.

[1249] A "meal kit" refers to a set of ingredients prepared based on a pre-designed menu.

[1250] "Food in the refrigerator" refers to the food stored in the user's refrigerator.

[1251] This invention is a system that provides an optimal menu considering the user's personal information and emotional data, and even delivers the meal kit. Specifically, the system operates in the following steps.

[1252] First, the user accesses the system's webpage or app using their device. The user enters their email address and password to create an account. The server receives the entered information, registers the new user information in the database, and then sends a confirmation email. The user clicks the link in the email to authenticate their account. This completes the account creation process.

[1253] Next, the user enters personal information from their device, such as family structure, preferences, allergy information, and special diet needs. This information is sent to the server and stored in the database. For example, if the family consists of two adults and one infant, preferences are for Japanese food, and allergies include peanuts, this data will be stored in the user's profile.

[1254] The server uses stored personal information to call up an AI model (such as OpenAI's GPT-4) to generate an optimal menu. For example, if a family consists of two adults and one infant and prefers Japanese food, the AI ​​will suggest a menu such as "teriyaki chicken," "spinach with sesame dressing," and "rice." The server saves the generated menu in a database and presents it to the user via the terminal.

[1255] Users can review the menu and select special meals (e.g., infant menus). The selected information is sent to the server, which then prepares a meal kit tailored to the special meal. For example, the server might prepare a meal kit of "Teriyaki Chicken" with the seasoning modified for infants and include "Soft Spinach."

[1256] Next, the user selects a delivery method using a terminal. Options include "home delivery" and "refrigerated locker at the nearest supermarket." If the user selects "home delivery," that information is sent to the server. The server then instructs the logistics system to deliver the meal kit to the location specified by the user.

[1257] Furthermore, the user takes pictures of the food in the refrigerator with their device's camera and uploads the images to the server. The server uses an image analysis engine (such as the Google Cloud Vision API) to recognize the food. Based on the recognized food, the server uses an auxiliary menu generation method to generate the optimal menu and presents it to the user. For example, if the refrigerator contains cabbage, carrots, and pork, menus such as "stir-fried pork and cabbage" and "carrot salad" will be suggested.

[1258] Users input their current mood and emotions through their device. For example, they might input "I'm feeling stressed" or "I'm tired." The server adjusts the menu based on the emotional input data. For instance, if the user is feeling stressed, a relaxing dessert such as a "matcha-flavored dessert" might be suggested.

[1259] The server continuously collects user emotional data and provides even more personalized suggestions based on long-term preference analysis. For example, if a user previously preferred a particular menu item when they were feeling stressed, this information is used to optimize future suggestions.

[1260] Specific example:

[1261] The user enters their family composition (2 adults, 1 infant), preferences (Japanese food), and allergies (peanuts) into the terminal. The server receives this information, saves it to a database, and then calls an AI model to generate menus such as "Teriyaki Chicken," "Spinach with Sesame Dressing," and "Rice." The user reviews the suggestions on the terminal and selects a special meal (for the infant). Based on this information, the server prepares a meal kit, and if the user selects "Home Delivery," the meal kit is delivered. Additionally, if the user enters "I'm feeling stressed," the server will suggest additional menu items such as a relaxing "Matcha-flavored dessert."

[1262] Example of a prompt:

[1263] The user has entered their family structure (2 adults, 1 infant) and preferred Japanese food. Peanut allergy information has also been provided. Please suggest the most suitable menu and delivery options.

[1264] Thus, the system of the present invention provides personalized menu suggestions and meal kit delivery services tailored to the user's needs and emotions, thereby reducing the burden of daily meal preparation and improving the quality of life.

[1265] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1266] Step 1:

[1267] User account creation

[1268] The user accesses the system's webpage or app using their device. The user enters their email address and password and submits the account creation form. The server receives this information and registers the new user information in its database. The server sends a confirmation email to the registered email address. The user clicks the link in the email to authenticate their account, completing the account creation process.

[1269] Input: User's email address, password

[1270] Output: Account creation confirmation email

[1271] Step 2:

[1272] Input and storage of user personal information

[1273] Users use their devices to input personal information such as family structure, preferences, allergy information, and special dietary needs. The entered information is sent to the server, which stores it in a database. For example, if a user inputs that their family consists of two adults and one infant, their preferences are for Japanese food, and their allergy is peanuts, that information will be saved in the user's profile.

[1274] Input: Family composition, preferences, allergy information, special diet needs

[1275] Output: Personal information stored in the database

[1276] Step 3:

[1277] AI-powered optimal menu generation

[1278] The server uses stored personal information to call the appropriate AI model (e.g., a generative AI model). The AI ​​generates an optimal menu based on the personal information and emotional data. For example, if the family consists of two adults and one infant, and their preference is for Japanese food, the AI ​​will suggest a menu such as "teriyaki chicken," "spinach with sesame dressing," and "rice." The server saves the generated menu in a database.

[1279] Input: Personal Information

[1280] Output: Optimal menu

[1281] Step 4:

[1282] User-selected menu

[1283] The user reviews the suggested menu through their device. If a special diet (e.g., infant menu) is selected, the selection information is sent to the server, which stores it in a database. The process of generating a meal kit based on the special diet information is then initiated.

[1284] Input: Suggested menu, selection of special meals

[1285] Output: Special diet information stored on the server

[1286] Step 5:

[1287] Production of meal kits based on special diets

[1288] The server generates optimized meal kits based on the user's selected special diet information. For example, it might prepare a meal kit for infants with modified seasonings for "teriyaki chicken" and include "soft spinach." This information is managed by the server and used to integrate with the logistics system.

[1289] Input: Special diet information

[1290] Output: Optimized meal kit information

[1291] Step 6:

[1292] User selection of delivery method and delivery of meal kits

[1293] The user selects a delivery method using a terminal. Options include "home delivery" and "refrigerated locker at the nearest supermarket." If the user selects "home delivery," that information is sent to the server. The server then instructs the logistics system, and the meal kit is delivered to the location specified by the user.

[1294] Input: Select shipping method

[1295] Output: Delivery Instructions

[1296] Step 7:

[1297] Image recognition of food items inside a refrigerator

[1298] The user takes a picture of the food in their refrigerator with their camera and uploads the image from their device to the server. The server uses an image analysis engine (e.g., an image analysis API) to recognize the food in the image. Based on the recognized food, the server generates an optimal menu and presents it to the user.

[1299] Input: Image of food items in the refrigerator

[1300] Output: Optimal menu

[1301] Step 8:

[1302] User emotional input and menu adjustment

[1303] Users input their current mood and emotions through their device, such as "I'm stressed" or "I'm tired." The device sends this information to a server. The server adjusts the menu based on the emotional input data. For example, if the user is feeling stressed, a relaxing dessert such as a "matcha-flavored dessert" might be suggested.

[1304] Input: Current mood or feelings

[1305] Output: Adjusted menu

[1306] Step 9:

[1307] Accumulation of emotional data and long-term preference analysis

[1308] The server continuously accumulates user emotional data. This stored emotional data is used for long-term preference analysis. This allows the system to learn what emotions users have experienced and which menu items they have preferred in the past, enabling more personalized suggestions. For example, if a user has preferred a particular menu item when feeling stressed in the past, this information can be used to optimize future suggestions.

[1309] Input: Sentiment data

[1310] Output: Personalized suggestions

[1311] (Application Example 2)

[1312] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1313] In modern dietary habits, it is difficult for users to create personalized menus tailored to their family structure, preferences, health status, and even their emotions, and then quickly prepare meal kits based on those menus. Furthermore, there is no consistent system for efficiently utilizing ingredients in the refrigerator while providing meals that meet individual needs. As a result, users often experience stress and inconvenience.

[1314] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes an input means for inputting the user's personal information, a menu generation means for generating a suitable menu based on the personal information, a presentation means for presenting the generated menu to the user, a preparation means for preparing a meal kit corresponding to the selected special diet, a delivery means for delivering the meal kit to the user, an emotion input means for inputting or recognizing the user's emotions, and an emotion adjustment means for adjusting the menu based on the emotion input means. This enables personalized menu suggestions that respond to the user's individual needs and emotions, as well as efficient delivery of meal kits.

[1315] "User personal information" refers to information related to an individual user, such as family structure, age group, preferences, allergy information, and special dietary needs.

[1316] An "input method" refers to an interface or device that allows a user to input personal information or emotional data into a system.

[1317] A "menu generation method" is a means for generating an optimal menu based on the user's personal information and emotional data.

[1318] "Presentation means" refers to means of presenting the generated menu to the user visually or audibly.

[1319] "Preparation methods" refer to the means of preparing meal kits that correspond to the selected special diet.

[1320] "Delivery method" refers to the means by which prepared meal kits are delivered to users.

[1321] An "emotion input method" is a means of inputting or recognizing a user's emotions.

[1322] "Emotion adjustment methods" refer to methods for adjusting menus based on the user's emotional data.

[1323] "Image analysis means" refers to a means for analyzing images of food items inside a refrigerator that have been uploaded by a user.

[1324] The "supplementary menu generation means" is a means for generating supplementary menus based on ingredients recognized by the image analysis means.

[1325] "Auxiliary presentation means" refers to a means for presenting menus generated by the auxiliary menu generation means to the user.

[1326] This invention is a system that generates menus and provides / delivers meal kits, taking into account the user's family structure, preferences, health status, and emotions. The system operates when the user creates an account via smartphone, enters personal information, uploads images of ingredients in their refrigerator, or enters their emotions.

[1327] First, users create an account using their smartphone. They enter their email address and password to create the account. Next, they enter personal information such as family structure, age group, preferences, allergy information, and special dietary needs. This information is sent from the device to the server and stored in the database.

[1328] After personal information is saved, the server uses AI to generate an optimal menu based on the saved information. For example, if the family consists of two adults and one infant, and their preference is for Japanese food, a menu such as "Teriyaki Chicken," "Spinach with Sesame Dressing," and "Rice" will be generated. Furthermore, the menu is adjusted using an emotion input method that allows the user to input or recognize their emotions. If the user inputs that they are feeling stressed, the menu will suggest dishes using ingredients with relaxing effects or their favorite comfort foods.

[1329] The generated menu is presented to the user via a terminal. The user can select a special meal (for example, a menu for infants) from the presented menu. When the user selects a special meal, that information is sent to the server, which generates a meal kit corresponding to the special meal. For example, for a family with an infant, a meal kit is prepared that includes teriyaki chicken seasoned for infants and soft spinach. Next, the user selects a delivery method and chooses "home delivery," at which point the meal kit is delivered to the user's home.

[1330] Furthermore, users can take photos of the ingredients in their refrigerator and upload the images to the system. The device sends the images to the server, which then performs image analysis. For example, if the refrigerator contains cabbage, carrots, and pork, the server recognizes these ingredients and generates menus such as "stir-fried pork and cabbage" and "carrot salad." The generated menus are then presented to the user via the device.

[1331] The hardware and software used include user terminals (e.g., smartphones), image recognition technology (e.g., OpenCV, TensorFlow), and emotion recognition technology (e.g., EmotionRecognition model). Menus are generated using AI, and deep learning models (e.g., TensorFlow or PyTorch) are used on the server side.

[1332] For example, if a user enters their family structure (2 adults, 1 infant), preferences (likes Japanese food), and allergies (peanuts), the system will suggest "Teriyaki Chicken," "Sesame Spinach," and "Rice." Furthermore, if the user selects the "Infant Menu" and specifies "Home Delivery," the meal kit will be delivered to the user's home. Also, if the user enters that they are "feeling stressed," the system will suggest additional items such as a "Matcha-flavored dessert" that has a relaxing effect.

[1333] Examples of specific prompt statements include the following:

[1334] "Based on the following profile and emotional data, please suggest the following menu: User Profile: Family Composition: 2 adults, 1 infant; Preferences: Japanese food; Allergies: Peanuts; Special Diet Needs: Infant menu; Emotions: Feeling stressed."

[1335] By fulfilling these requirements, the system can provide personalized menu suggestions and meal kit delivery services tailored to the user's needs and emotions.

[1336] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1337] Step 1:

[1338] The user creates an account using their smartphone. The input consists of an email address and password, which the device receives and sends to the server. The server receives this information, registers the user account in its database, and sends a confirmation email. The output is a confirmation email indicating successful account creation.

[1339] Step 2:

[1340] The user enters personal information such as family structure, age group, preferences, allergy information, and special dietary needs. This information is sent from the terminal to the server, which stores this data in a database. The output confirms the saving of the personal information.

[1341] Step 3:

[1342] The server uses AI to generate the optimal menu based on stored personal information. In this process, personal information stored in a database is used as input, and a generation AI model is used to calculate and generate the menu. The output is the generated menu data.

[1343] Step 4:

[1344] The terminal displays the generated menu to the user. The input is menu data received from the server, which the terminal displays to the user. The output is the menu presented to the user.

[1345] Step 5:

[1346] The user selects a special meal from the presented menu. The user's selection is sent from the terminal to the server, which verifies this information. The input is the user's selected special meal information, and the server prepares the meal kit based on it. The output is the data for the prepared meal kit.

[1347] Step 6:

[1348] The user selects a shipping method. The user's selected shipping method is sent from the terminal to the server, which then issues instructions to the logistics system. The input is the user's selected shipping method, and the output is a notification that the shipping arrangements have been completed.

[1349] Step 7:

[1350] The user takes a picture of the food in their refrigerator and uploads the image from their device to the server. The server uses image analysis to recognize the food. The input is an image of the food in the refrigerator, and the output is the analyzed food data.

[1351] Step 8:

[1352] The server generates supplementary menus based on recognized ingredient data. In this process, the analyzed ingredient data is used as input, and a generation AI model is used to calculate and generate the supplementary menus. The output is the data of the generated supplementary menus.

[1353] Step 9:

[1354] The terminal displays the generated supplementary menu to the user. The input is the supplementary menu data received from the server, which the terminal displays to the user. The output is the supplementary menu presented to the user.

[1355] Step 10:

[1356] The user inputs an emotion or uploads a facial image. The device sends this data to the server, which uses the emotion input method to recognize the emotion. The input is the user's emotion or facial image data, and the output is the recognized emotion data.

[1357] Step 11:

[1358] The server adjusts the menu based on recognized emotion data. This process uses emotion data as input and readjusts the menu using emotion adjustment mechanisms. The output is the adjusted menu data.

[1359] This processing flow enables the delivery of meal kits and menu suggestions optimized to the individual needs and emotions of each user.

[1360] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1361] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1362] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1363] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1364] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1365] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1366] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1367] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1368] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1369] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1370] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1371] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1372] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1373] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1374] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1375] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1376] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1377] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1378] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1379] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1380] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1381] The following is further disclosed regarding the embodiments described above.

[1382] (Claim 1)

[1383] An input method for entering the user's personal information,

[1384] A menu generation means that generates a suitable menu based on the aforementioned personal information,

[1385] A means of presenting the generated menu to the user,

[1386] Preparation methods for preparing meal kits that correspond to the selected special diet,

[1387] A delivery means for delivering the meal kit to the user,

[1388] A system that includes this.

[1389] (Claim 2)

[1390] An image analysis method that analyzes images of food items in a refrigerator uploaded by a user,

[1391] An auxiliary menu generation means that generates a menu based on the ingredients recognized by the image analysis means,

[1392] An auxiliary presentation means for presenting menus generated by the auxiliary menu generation means to the user,

[1393] The system according to claim 1, further comprising:

[1394] (Claim 3)

[1395] The system according to claim 1, comprising means for using evaluation data to further optimize menus based on the user's past menu evaluation data.

[1396] "Example 1"

[1397] (Claim 1)

[1398] Input methods for entering the user's personal information,

[1399] A meal plan generation means that generates an optimal meal plan based on the aforementioned personal information,

[1400] A means of presenting the generated meal plan to the user,

[1401] Preparation methods for preparing ingredient kits corresponding to selected specific foods,

[1402] A delivery method for delivering the aforementioned food kit to the user,

[1403] A system that includes this.

[1404] (Claim 2)

[1405] An image analysis means for analyzing images of food items in a storage room uploaded by a user,

[1406] A supplementary meal plan generation means generates a supplementary meal plan based on the ingredients identified by the image analysis means,

[1407] A supplementary presentation means for presenting the meal plan generated by the aforementioned supplementary meal plan generation means to the user,

[1408] The system according to claim 1, further comprising:

[1409] (Claim 3)

[1410] The system according to claim 1, comprising evaluation data utilization means for further optimizing a meal plan based on the user's past meal plan evaluation data.

[1411] "Application Example 1"

[1412] (Claim 1)

[1413] An input method for entering the user's personal information,

[1414] A menu generation means that generates a suitable menu based on the aforementioned personal information,

[1415] A means of presenting the generated menu to the user,

[1416] Preparation methods for preparing meal kits that correspond to the selected special diet,

[1417] A delivery means for delivering the meal kit to the user,

[1418] An image analysis method that uploads images of food items inside a refrigerator and analyzes those images,

[1419] A supplementary menu generation means that generates a supplementary menu based on the ingredients recognized by the image analysis means,

[1420] A supplementary presentation means for presenting supplementary menus generated by the supplementary menu generation means to the user,

[1421] A means of ordering food based on the provided menu,

[1422] A system that includes this.

[1423] (Claim 2)

[1424] An image analysis method that analyzes images of food items in a refrigerator uploaded by a user,

[1425] An auxiliary menu generation means that generates a menu based on the ingredients recognized by the image analysis means,

[1426] An auxiliary presentation means for presenting menus generated by the auxiliary menu generation means to the user,

[1427] The system according to claim 1, further comprising:

[1428] (Claim 3)

[1429] The system according to claim 1, comprising means for using evaluation data to further optimize menus based on the user's past menu evaluation data.

[1430] "Example 2 of combining an emotion engine"

[1431] (Claim 1)

[1432] An input method for entering the user's personal information,

[1433] A menu generation means that generates a suitable menu based on the aforementioned personal information and user sentiment data,

[1434] A presentation means for presenting the generated menu to the user,

[1435] A means of preparing meal kits that correspond to the special diet selected by the user,

[1436] A delivery method for delivering the meal kit to the user,

[1437] A system that includes this.

[1438] (Claim 2)

[1439] An image analysis method that analyzes images of food items in a refrigerator uploaded by a user,

[1440] An auxiliary menu generation means that generates a menu based on the ingredients recognized by the image analysis means,

[1441] A supplementary presentation means for presenting the generated supplementary menu to the user,

[1442] The system according to claim 1, including the following:

[1443] (Claim 3)

[1444] The system according to claim 1, comprising evaluation data utilization means for further optimizing menus based on the user's past menu evaluation data and sentiment data.

[1445] "Application example 2 when combining with an emotional engine"

[1446] (Claim 1)

[1447] An input method for entering the user's personal information,

[1448] A menu generation means that generates a suitable menu based on the aforementioned personal information,

[1449] A means of presenting the generated menu to the user,

[1450] Preparation methods for preparing meal kits that correspond to the selected special diet,

[1451] A delivery means for delivering the meal kit to the user,

[1452] An emotion input means for inputting or recognizing the user's emotions,

[1453] An emotion adjustment means that adjusts the menu based on the emotion input means,

[1454] A system that includes this.

[1455] (Claim 2)

[1456] An image analysis method that analyzes images of food items in a refrigerator uploaded by a user,

[1457] An auxiliary menu generation means that generates a menu based on the ingredients recognized by the image analysis means,

[1458] An auxiliary presentation means for presenting menus generated by the auxiliary menu generation means to the user,

[1459] The system according to claim 1, further comprising:

[1460] (Claim 3)

[1461] The system according to claim 1, comprising means for using evaluation data to further optimize menus based on the user's past menu evaluation data. [Explanation of Symbols]

[1462] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. An input method for entering the user's personal information, A menu generation means that generates a suitable menu based on the aforementioned personal information, A means of presenting the generated menu to the user, Preparation methods for preparing meal kits that correspond to the selected special diet, A delivery means for delivering the meal kit to the user, A system that includes this.

2. An image analysis method that analyzes images of food items in a refrigerator uploaded by a user, An auxiliary menu generation means that generates a menu based on the ingredients recognized by the image analysis means, An auxiliary presentation means for presenting menus generated by the auxiliary menu generation means to the user, The system according to claim 1, further comprising:

3. The system according to claim 1, comprising evaluation data utilization means for further optimizing menus based on the user's past menu evaluation data.

Images