system

Abstract

The system according to this embodiment aims to provide menu suggestions and automatically order ingredients, taking into account the user's food preferences and health condition. [Solution] The system according to the embodiment comprises a collection unit, a suggestion unit, an order unit, a delivery unit, a feedback unit, an improvement unit, a dialogue unit, and an assistant unit. The collection unit collects information such as the user's food preferences, allergies, health status, budget, and cooking skills. The suggestion unit suggests a week's worth of menus based on the information collected by the collection unit. The order unit generates an ingredient list based on the menu suggested by the suggestion unit and places an automatic order. The delivery unit provides detailed recipes. The feedback unit receives feedback from the user. The improvement unit improves the suggestions based on the feedback received by the feedback unit. The dialogue unit allows the menu to be adjusted with simple instructions. The assistant unit has an agent that acts as an assistant during cooking.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology, menu proposals and automatic ordering of food materials considering the user's food preferences and health conditions are not sufficiently carried out, and there is room for improvement.

[0005] The system according to the embodiment aims to make menu proposals and automatically order food materials considering the user's food preferences and health conditions.

Means for Solving the Problems

[0006] The system according to this embodiment comprises a data collection unit, a suggestion unit, an order unit, a delivery unit, a feedback unit, an improvement unit, a dialogue unit, and an assistant unit. The data collection unit collects information such as the user's food preferences, allergies, health status, budget, and cooking skills. The suggestion unit suggests a week's worth of menus based on the information collected by the data collection unit. The order unit generates an ingredient list based on the menu suggested by the suggestion unit and places an automatic order. The delivery unit provides detailed recipes. The feedback unit receives feedback from the user. The improvement unit improves the suggestions based on the feedback received by the feedback unit. The dialogue unit allows the user to adjust the menu with simple instructions. The assistant unit has an agent that acts as an assistant during cooking. [Effects of the Invention]

[0007] The system according to this embodiment can suggest menus that take into account the user's food preferences and health condition, and automatically order ingredients. [Brief explanation of the drawing]

[0008] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8]This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Modes for carrying out the invention]

[0009] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0010] First, let's explain the terminology used in the following explanation.

[0011] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit).

[0012] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0013] In the following embodiments, the tagged storage is one or more non-volatile storage devices that store various programs, various parameters, and the like. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0014] In the following embodiments, the tagged communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0016] [First Embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. Also, the database 24 and the communication I / F 26 are connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. Also, the reception device 38, the output device 40, and the camera 42 are connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by contact of an indicator (e.g., a pen or a finger, etc.) by detecting the contact of the indicator. The microphone 38B receives user input by voice by detecting the voice of the user. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, a specific processing unit 290 (see FIG. 2) acquires data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user by outputting the data in a form perceptible to the user (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0023] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0025] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0026] In the smart device 14, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0027] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device having the data generation model 58. The data processing device 12 may also be a server device or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example of form 1) The AI ​​agent system according to an embodiment of the present invention is a system that proposes a week's worth of menus, taking into account the user's food preferences, allergies, health status, budget, and cooking skills. This system collects information such as the user's food preferences, allergies, health status, budget, and cooking skills, and the AI ​​proposes a week's worth of menus based on the collected information. Furthermore, it generates a list of ingredients necessary for the proposed menu and automatically places orders with affiliated e-commerce sites. It also provides detailed recipes and suggestions that take cleanup into consideration. In addition, it is equipped with a function to suggest meal prepping for the weekend, a mechanism for receiving user feedback and an improvement cycle, an interactive function that allows menu adjustments with simple instructions (conversation), a function in which the agent acts as an assistant during cooking, autonomous ingredient management, health monitoring integration, smart kitchen integration, community planning, personalized nutrition guidance, social integration and sharing agent, and a world cuisine planner function. Furthermore, a "personal recipe advisor" function has been added. For example, it collects information such as the user's food preferences, allergies, health status, budget, and cooking skills. This information is either entered by the user or collected from past data. For example, a user provides information such as "I like spicy food," "I have a dairy allergy," "I'm trying to lose weight," "My budget is 5,000 yen per week," and "I'm a beginner cook." Next, based on the collected information, the AI ​​suggests a week's worth of meal plans. The AI ​​generates a balanced menu considering the user's preferences, allergies, health condition, budget, and cooking skills. For example, it might suggest "Teriyaki Chicken and Stir-fried Vegetables" for Monday, "Grilled Fish and Salad" for Tuesday, and "Tofu Steak and Vegetable Soup" for Wednesday. Furthermore, it generates a list of ingredients needed for the suggested menu and automatically places orders with partner e-commerce sites. This allows users to obtain the necessary ingredients without the hassle of purchasing them themselves. For example, an ingredient list such as "200g of chicken," "1 head of cabbage," and "2 carrots" is generated and ordered from the e-commerce site. The system also provides detailed recipes and suggestions that take cleanup into consideration. The AI ​​provides step-by-step guides so that even beginner cooks can easily prepare meals. It also suggests menus that minimize the use of equipment and cooking processes, reducing the burden of cleanup.For example, it suggests menus that can be made in one pan or require minimal washing up. It also has a feature that suggests meal prepping for the weekend. The AI ​​suggests menus that can be prepared in advance on the weekend, providing easy-to-eat meals for busy weekdays. For example, it might suggest "prepare curry on the weekend and reheat it on weekdays." It also has a system for receiving user feedback and an improvement cycle. Users can leave ratings and comments on menus and recipes, and the AI ​​uses this feedback to improve the quality of its suggestions. For example, it receives feedback such as "This recipe was delicious" or "This menu is too time-consuming" and incorporates it into future suggestions. It also has an interactive function that allows users to adjust menus with simple instructions (conversation). Users can give simple instructions to menus and recipes through voice input or chatbot-style conversation. For example, it responds in real time to instructions such as "I don't have much time today, so change it to a menu that can be made quickly" or "I don't like spicy food, so change it to a different menu." It also has a function where an agent acts as an assistant during cooking. The AI ​​guides the cooking process step by step with voice and text, supporting even beginners so that they can cook without getting lost. It also offers a timer function, visual support, question answering, and a voice assistant function. For example, it can respond to instructions such as "Tell me the next step" or "Set the timer to 5 minutes." It also features autonomous food management, health monitoring integration, smart kitchen integration, community planning, personalized nutrition guidance, social integration and sharing agent, and a World Cuisine Planner function. For example, it uses internet-connected sensors to automatically manage inventory in the refrigerator and pantry and automatically orders any missing ingredients from e-commerce sites. It also integrates with health data from wearable devices to adjust menus based on health status. Furthermore, it integrates with smart kitchen devices to autonomously control the cooking process. It also provides community-based meal planning, personalized nutrition guidance, social integration and sharing agent, and a World Cuisine Planner function. Finally, it adds a "Personal Recipe Advisor" function.The AI ​​learns the user's preferences and frustrations, and suggests personalized recipes. For example, it might suggest, "This user likes spicy food and is likely to like this combination of ingredients." It also accumulates user feedback to improve the accuracy of its suggestions. As a result, the AI ​​agent system can suggest a week's worth of meal plans, generate a list of necessary ingredients, and automatically place orders, taking into account the user's food preferences, allergies, health condition, budget, and cooking skills.

[0029] The AI ​​agent system according to this embodiment comprises a collection unit, a suggestion unit, an order unit, a delivery unit, a feedback unit, an improvement unit, a dialogue unit, and an assistant unit. The collection unit collects information such as the user's food preferences, allergies, health status, budget, and cooking skills. The collection unit collects information from, for example, information entered by the user and past data. For example, the collection unit receives information from the user such as "I like spicy food," "I have a dairy allergy," "I'm trying to lose weight," "My budget is 5,000 yen per week," and "I'm a beginner cook." The suggestion unit proposes a week's worth of menus based on the information collected by the collection unit. The suggestion unit uses, for example, AI to generate a balanced menu that takes into account the user's preferences, allergies, health status, budget, and cooking skills. For example, the suggestion unit proposes menus for each day, such as "Teriyaki chicken and stir-fried vegetables" on Monday, "Grilled fish and salad" on Tuesday, and "Tofu steak and vegetable soup" on Wednesday. The order unit generates a list of ingredients based on the menu proposed by the suggestion unit and places an automatic order. The ordering unit generates a list of ingredients needed for a suggested menu and automatically places orders with partner e-commerce sites. For example, the ordering unit generates an ingredient list such as "200g chicken," "1 cabbage," and "2 carrots" and places orders with e-commerce sites. The delivery unit provides detailed recipes. For example, the delivery unit provides step-by-step guides so that even beginner cooks can easily prepare the meals. For example, the delivery unit suggests menus that minimize the amount of equipment used and the cooking process, reducing the burden of cleanup. The feedback unit receives feedback from users. For example, the feedback unit allows users to leave ratings and comments on menus and recipes. For example, the feedback unit receives feedback such as "This recipe was delicious" or "This menu is too time-consuming." The improvement unit improves the suggestions based on the feedback received by the feedback unit. For example, the improvement unit improves the quality of suggestions based on the feedback. For example, the improvement unit incorporates the feedback into future suggestions. The dialogue unit allows users to adjust menus with simple instructions (conversation). The dialogue unit can, for example, give simple instructions regarding menus and recipes through voice input or a chatbot-style conversation.For example, the dialogue unit responds in real time to instructions such as, "I don't have much time today, so please change the menu to one that can be prepared quickly," or "I don't like spicy food, so please make a different menu." The assistant unit has an agent that acts as an assistant during cooking. The assistant unit, for example, guides the cooking process step by step with voice and text, supporting even beginners so that they can cook without getting lost. For example, the assistant unit responds to instructions such as, "Tell me the next step," or "Set the timer for 5 minutes." As a result, the AI ​​agent system according to this embodiment can propose a week's worth of menus, generate a list of necessary ingredients, and automatically order them, taking into account the user's food preferences, allergies, health condition, budget, cooking skills, etc.

[0030] The data collection unit collects information such as the user's food preferences, allergies, health status, budget, and cooking skills. For example, the unit collects information from user input and past data. Specifically, it collects information on the user's food preferences and allergies based on the information the user enters into the application. For instance, the user might provide information such as "I like spicy food," "I have a dairy allergy," "I'm trying to lose weight," "My budget is 5,000 yen per week," or "I'm a beginner cook." This information is saved as the user's profile and used for future suggestions and orders. The data collection unit also analyzes the user's past order history, ratings, and feedback to understand their preferences and tendencies. For example, it identifies the types of food and seasonings the user prefers based on ratings and comments on previously ordered dishes. Furthermore, the data collection unit also collects information on the user's health status. For example, if the user is linked to a health management app, the data is acquired to understand the user's health. This allows the data collection unit to suggest menus tailored to the user's health condition. The data collection unit centrally manages this information and provides it to the suggestion and ordering units. This allows the data collection unit to efficiently collect information that meets the diverse needs of users, thereby improving the overall accuracy and usability of the system.

[0031] The suggestion department proposes a week's worth of menus based on the information collected by the data collection department. The suggestion department uses AI, for example, to generate balanced menus that take into account the user's preferences, allergies, health condition, budget, and cooking skills. Specifically, the AI ​​analyzes the collected information and generates the optimal menu based on the user's preferences and constraints. For example, the suggestion department might suggest menus for each day, such as "Teriyaki Chicken and Stir-fried Vegetables" for Monday, "Grilled Fish and Salad" for Tuesday, and "Tofu Steak and Vegetable Soup" for Wednesday. The AI ​​also considers nutritional balance, calories, and cooking time when generating menus, allowing users to lead a healthy and efficient diet. Furthermore, the suggestion department improves the accuracy of its suggestions by incorporating user feedback. For example, if a user provides feedback such as "This recipe was delicious" or "This menu is too time-consuming," the suggestion department uses that information to improve its next suggestion. In this way, the suggestion department can provide the optimal menu that meets the user's needs and increase user satisfaction.

[0032] The ordering department generates ingredient lists and automatically places orders based on menus proposed by the suggestion department. For example, the ordering department generates an ingredient list necessary for a proposed menu and automatically places orders with partner e-commerce sites. Specifically, the ordering department lists the ingredients needed for each menu item in the proposed menu and places the optimal order considering the user's budget and inventory status. For example, the ordering department generates an ingredient list such as "200g chicken," "1 cabbage," and "2 carrots," and places an order with a partner e-commerce site. The ordering department manages the user's account information and delivery address information to ensure a smooth ordering process. The ordering department also monitors the order status and delivery status in real time and notifies the user. This allows the user to understand the progress of their order and receive ingredients with peace of mind. Furthermore, the ordering department can analyze the user's past order history and suggest repeat orders and subscriptions. In this way, the ordering department can improve user convenience and support efficient ingredient procurement.

[0033] The service provider offers detailed recipes. For example, they provide step-by-step guides so that even beginners can easily cook. Specifically, they suggest menus that minimize the amount of equipment and cooking process used, reducing the burden of cleanup. For instance, in the "Teriyaki Chicken" recipe, they provide detailed instructions from preparing the chicken to cooking and plating. Each step includes information on necessary equipment, cooking time, and precautions, allowing even beginners to cook without hesitation. The service provider also offers troubleshooting and advice during cooking. For example, they provide specific advice such as "reduce the heat if the chicken is about to burn" or "add water if the sauce is too thick." In this way, the service provider supports users so that they can cook with confidence and create delicious meals. Furthermore, the service provider also suggests variations and adaptations of recipes. For example, they offer methods for adapting "Teriyaki Chicken" into "Teriyaki Chicken Rice Bowl" or provide variations using different ingredients. In this way, the service provider helps users broaden their culinary horizons and increase their enjoyment of meals.

[0034] The Feedback Department receives feedback from users. For example, users can leave ratings and comments on menus and recipes. Specifically, the Feedback Department provides rating forms and comment sections within the application to make it easy for users to provide feedback. For example, users can enter ratings and comments such as "This recipe was delicious" or "This menu is too time-consuming." The Feedback Department collects this feedback and stores it in a database. The Feedback Department also analyzes user ratings and comments and provides them to the Suggestion Department and Improvement Department. This allows the Feedback Department to support service improvements that reflect user opinions. Furthermore, the Feedback Department can also provide replies and advice to user feedback. For example, it can reply with "I'm glad you liked this recipe" or "Please try this menu next time." This strengthens communication with users and increases user satisfaction.

[0035] The Improvement Department improves proposals based on feedback received by the Feedback Department. For example, the Improvement Department improves the quality of proposals based on feedback. Specifically, the Improvement Department analyzes user feedback and reflects it in the Proposal Department and the Delivery Department. For example, if a user provides feedback such as "This menu is too time-consuming," the Improvement Department will consider ways to simplify the cooking process and reflect this in the next proposal. The Improvement Department also understands user preferences and trends to provide more personalized proposals. For example, if a user provides feedback such as "I like spicy food," the Improvement Department will include more spicy dishes in the next proposal. In this way, the Improvement Department can provide optimal proposals that meet user needs and increase user satisfaction. Furthermore, the Improvement Department regularly reports the results of its feedback analysis and uses them to improve the entire system. In this way, the Improvement Department can achieve continuous service improvement and provide a more valuable system for users.

[0036] The dialogue unit can adjust menus with simple instructions (conversation). For example, the dialogue unit can give simple instructions regarding menus and recipes through voice input or chatbot-style dialogue. Specifically, when a user gives instructions by voice, such as "I don't have much time today, so change it to a menu that can be made quickly" or "I don't like spicy food, so please make a different menu," the AI ​​analyzes the instructions and adjusts the menu in real time. The dialogue unit uses natural language processing technology to accurately understand the user's instructions and respond appropriately. The dialogue unit also supports chatbot-style dialogue, allowing users to input instructions in text. For example, if a user inputs instructions such as "Review this week's menu" or "Suggest a new recipe," the dialogue unit will adjust the menu based on those instructions. This allows the dialogue unit to respond flexibly to the user's needs and improve user convenience. Furthermore, the dialogue unit saves the history of conversations with the user and uses it for future conversations. This allows the dialogue unit to understand the user's preferences and tendencies and provide more personalized responses.

[0037] The Assistant Unit acts as an assistant during cooking. For example, the Assistant Unit guides the cooking process step by step with voice and text, supporting even beginners so they can cook without getting lost. Specifically, when the user gives instructions such as "Tell me the next step" or "Set the timer for 5 minutes," the AI ​​provides appropriate guidance in response to those instructions. For example, it provides specific instructions in voice and text such as "Put the chicken in the frying pan and cook over medium heat for 5 minutes" and "Next, add the cabbage and sauté for another 3 minutes." The Assistant Unit also handles troubleshooting during cooking, providing appropriate advice to questions such as "What should I do if the sauce looks like it's going to burn?" In this way, the Assistant Unit supports users so they can cook with peace of mind. Furthermore, the Assistant Unit monitors the progress of cooking in real time and sets the timer or provides guidance on the next step as needed. In this way, the Assistant Unit can improve the user's cooking experience and support efficient and enjoyable cooking.

[0038] The collection unit is equipped with an autonomous food management agent. For example, the collection unit automatically manages the inventory of refrigerators and pantries using internet-connected sensors. For example, the collection unit automatically orders running low on ingredients from e-commerce sites. The collection unit can also manage the expiration dates of ingredients and suggest menus that prioritize the use of ingredients nearing their expiration date. This allows the collection unit to autonomously manage ingredients and automatically order running low on ingredients from e-commerce sites.

[0039] The data collection unit includes a health monitoring and menu adjustment agent. For example, the data collection unit works with health data from wearable devices to adjust menus based on the user's health status. For instance, the data collection unit suggests low-calorie or high-protein menus based on the user's health condition. It can also suggest nutritionally balanced menus based on the user's health goals. This allows the data collection unit to adjust menus based on the user's health status.

[0040] The data collection unit is equipped with a smart kitchen integration agent. The data collection unit can, for example, work with smart kitchen devices to autonomously control the cooking process. For example, the data collection unit can work with smart ovens and smart stoves to automatically set cooking temperatures and times. The data collection unit can also work with smart refrigerators to monitor food inventory in real time. As a result, the data collection unit can work with smart kitchen devices to autonomously control the cooking process.

[0041] The data collection unit includes a community-based meal planning agent. For example, the data collection unit collects and shares meal plans from communities in which users participate. For example, the data collection unit suggests new menus to users based on the meal plans of community members. The data collection unit can also collect popular recipes within a community and provide them to users. This enables the data collection unit to perform community-based meal planning.

[0042] The data collection unit is equipped with a personalized nutrition guidance agent. The data collection unit provides individualized nutrition guidance based, for example, the user's health status and food preferences. For instance, it suggests an appropriate nutritional balance according to the user's health goals. The data collection unit can also analyze the user's dietary history and evaluate their nutrient intake. This enables the data collection unit to provide personalized nutrition guidance.

[0043] The data collection unit includes social media integration and a sharing agent. For example, the data collection unit collects information about dishes shared by users on social media. For example, the data collection unit collects information based on the content of posts from cooking accounts that users follow. For example, the data collection unit can also analyze trends in cooking communities that users participate in and collect relevant information. This enables the data collection unit to perform social media integration and sharing.

[0044] The data collection unit includes a world cuisine planner function. For example, the data collection unit collects information about cuisines from around the world and provides it to the user. For example, the data collection unit suggests dishes from countries or regions that the user is interested in. The data collection unit can also provide recipes using ingredients from specific countries or regions. In this way, the data collection unit can provide a world cuisine planner function.

[0045] The data collection unit includes a personal recipe advisor. For example, the data collection unit learns the user's preferences and frustrations and suggests personalized recipes. For instance, it might suggest recipes based on the user's preference for spicy food and likely enjoyment of a particular ingredient combination. The data collection unit can also accumulate user feedback to improve the accuracy of its suggestions. This allows the data collection unit to provide a personalized recipe advisor function.

[0046] The data collection unit analyzes the user's past eating history and selects the optimal information collection method. For example, the data collection unit collects relevant information based on the dishes the user has enjoyed eating in the past. For example, the data collection unit filters out information to be avoided based on the dishes the user has avoided in the past. For example, the data collection unit can also prioritize the collection of information about specific ingredients from the user's eating history. This allows the data collection unit to select the optimal information collection method based on the user's past eating history.

[0047] The data collection unit filters information based on the user's current health status and lifestyle. For example, if the user is on a diet, the unit prioritizes collecting low-calorie recipes. If the user has allergies, the unit collects recipes that do not contain allergens. If the user is busy, the unit can also collect recipes that can be prepared in a short amount of time. This allows the data collection unit to filter information based on the user's health status and lifestyle.

[0048] The data collection unit prioritizes collecting highly relevant information, taking into account the user's geographical location. For example, it collects recipes using local specialties from the area where the user lives. If the user is traveling, it collects recipes for local specialties from that region. If the user is interested in a particular region, it can also collect information about the cuisine of that region. This allows the data collection unit to prioritize collecting highly relevant information based on the user's geographical location.

[0049] The data collection unit analyzes the user's social media activity and collects relevant information during data collection. For example, the data collection unit collects information about dishes that the user has shared on social media. For example, the data collection unit collects information based on the content of posts from cooking accounts that the user follows. For example, the data collection unit can also analyze trends in cooking communities that the user participates in and collect relevant information. In this way, the data collection unit can collect relevant information based on the user's social media activity.

[0050] The suggestion function adjusts the level of detail in its suggestions based on the user's health condition. For example, it might suggest healthy recipes based on items identified in the user's health checkup. Or, if the user is on a diet, it might suggest recipes that take calories and nutritional balance into consideration. If the user has a specific illness, it might suggest meals suitable for that illness. This allows the suggestion function to adjust the level of detail in its suggestions based on the user's health condition.

[0051] The suggestion function applies different suggestion algorithms depending on the user's cooking skill level. For example, it suggests easy, foolproof recipes to beginners, slightly more elaborate recipes to intermediate cooks, and even challenging recipes to advanced cooks. This allows the suggestion function to apply different suggestion algorithms depending on the user's cooking skill level.

[0052] The suggestion function prioritizes suggestions based on the user's eating history. For example, it might prioritize suggesting dishes the user has enjoyed eating in the past. It might also exclude dishes the user has avoided in the past. The suggestion function can also prioritize suggesting recipes using specific ingredients based on the user's eating history. This allows the suggestion function to prioritize suggestions based on the user's eating history.

[0053] The suggestion function adjusts the order of suggestions based on the relevance of the user's meals. For example, it suggests recipes related to dishes the user has eaten in the past. For example, it suggests that the user avoid eating dishes containing certain ingredients consecutively. For example, it can suggest a balanced menu based on the user's meal history. This allows the suggestion function to adjust the order of suggestions based on the relevance of the user's meals.

[0054] The ordering department analyzes the user's past order history to select the optimal ordering method when an order is placed. For example, the ordering department suggests relevant ingredients based on ingredients the user has frequently ordered in the past. For example, the ordering department filters out ingredients to avoid based on ingredients the user has avoided in the past. For example, the ordering department can also prioritize collecting information about specific ingredients from the user's order history. This allows the ordering department to select the optimal ordering method based on the user's past order history.

[0055] The ordering system customizes orders based on the user's current lifestyle. For example, if the user is busy, the system prioritizes ordering ingredients that are easy to cook. If the user is on a diet, the system prioritizes ordering low-calorie ingredients. If the user has allergies, the system can also order ingredients that do not contain allergens. This allows the ordering system to customize orders based on the user's current lifestyle.

[0056] The ordering system selects the optimal ordering method when an order is placed, taking into account the user's geographical location. For example, the ordering system might order ingredients that are specialties of the region where the user lives. If the user is traveling, the ordering system might order ingredients for local specialties of that region. If the user is interested in a particular region, the ordering system might also order ingredients related to the cuisine of that region. This allows the ordering system to select the optimal ordering method based on the user's geographical location.

[0057] The ordering department analyzes the user's social media activity when an order is placed and suggests the order. For example, the ordering department might order ingredients related to a dish the user has shared on social media. For example, the ordering department might order ingredients based on posts from cooking accounts the user follows. For example, the ordering department could also analyze trends in cooking communities the user participates in and order related ingredients. This allows the ordering department to suggest orders based on the user's social media activity.

[0058] The service provider will provide detailed guides tailored to the user's cooking skill level when providing recipes. For example, they will provide step-by-step detailed guides for beginners, for example, guides that focus on the essentials for intermediate cooks, and guides that include suggestions for variations for advanced cooks. This allows the service provider to provide detailed guides tailored to the user's cooking skill level.

[0059] The service provider customizes recipes based on the user's health condition when providing them. For example, if a user is on a diet, the service provider will provide low-calorie recipes. If a user has a specific illness, the service provider will provide recipes suitable for that illness. The service provider can also provide healthy recipes based on items pointed out to the user during a health checkup. In this way, the service provider can customize recipes based on the user's health condition.

[0060] The service provider adjusts recipe content based on the user's eating history when providing recipes. For example, the service provider provides relevant recipes based on dishes the user has enjoyed eating in the past. For example, the service provider filters out recipes to avoid based on dishes the user has avoided in the past. For example, the service provider can also prioritize providing recipes related to specific ingredients based on the user's eating history. In this way, the service provider can adjust recipe content based on the user's eating history.

[0061] The service provider analyzes the user's social media activity when providing recipes and suggests recipe content accordingly. For example, the service provider may provide recipes related to dishes the user has shared on social media. For example, the service provider may provide recipes based on posts from cooking accounts the user follows. For example, the service provider may analyze trends in cooking communities the user participates in and provide relevant recipes. This allows the service provider to suggest recipe content based on the user's social media activity.

[0062] The feedback unit analyzes the user's past feedback history to select the optimal collection method when collecting feedback. For example, the feedback unit suggests relevant questions based on the feedback the user has provided in the past. For example, the feedback unit filters out questions to avoid based on the question formats the user has avoided in the past. For example, the feedback unit can also prioritize collecting feedback on specific items from the user's feedback history. This allows the feedback unit to select the optimal collection method based on the user's past feedback history.

[0063] The feedback unit selects the optimal collection method when collecting feedback, taking into account the user's geographical location. For example, the feedback unit collects feedback on local specialties in the area where the user lives. For example, if the user is traveling, the feedback unit collects feedback on local cuisine in that area. For example, if the user is interested in a particular region, the feedback unit can also collect feedback on the cuisine of that region. This allows the feedback unit to select the optimal collection method based on the user's geographical location.

[0064] The improvement department analyzes past user feedback to select the optimal improvement method during the improvement process. For example, the improvement department proposes relevant improvement methods based on feedback previously provided by users. For example, the improvement department filters out improvement methods to be avoided based on improvement methods that users have avoided in the past. For example, the improvement department can also prioritize suggesting improvement methods for specific items based on the user's feedback history. This allows the improvement department to select the optimal improvement method based on past user feedback.

[0065] The improvement unit selects the optimal improvement method when making improvements, taking into account the user's geographical location information. For example, the improvement unit may suggest improvement methods related to local specialties in the area where the user lives. For example, if the user is traveling, the improvement unit may suggest improvement methods related to local cuisine in that area. For example, if the user is interested in a particular region, the improvement unit may also suggest improvement methods related to the cuisine of that region. In this way, the improvement unit can select the optimal improvement method based on the user's geographical location information.

[0066] The dialogue unit analyzes the user's past dialogue history to select the optimal dialogue method during a conversation. For example, the dialogue unit suggests relevant dialogue methods based on the user's past conversations. For example, the dialogue unit filters out dialogue methods to be avoided based on dialogue methods the user has avoided in the past. For example, the dialogue unit can also prioritize suggesting dialogue methods related to specific items based on the user's dialogue history. In this way, the dialogue unit can select the optimal dialogue method based on the user's past dialogue history.

[0067] The dialogue unit selects the optimal dialogue method during a conversation, taking into account the user's geographical location. For example, the dialogue unit may provide a dialogue method about local specialties in the area where the user lives. For example, if the user is traveling, the dialogue unit may provide a dialogue method about local cuisine in that area. For example, if the user is interested in a particular region, the dialogue unit may also provide a dialogue method about the cuisine of that region. In this way, the dialogue unit can select the optimal dialogue method based on the user's geographical location.

[0068] The assistant unit analyzes the user's past cooking history to select the optimal assistance method during assistance. For example, the assistant unit suggests relevant assistance methods based on the user's past cooking experiences. For example, the assistant unit filters out assistance methods to be avoided based on cooking methods the user has avoided in the past. For example, the assistant unit can also prioritize suggesting assistance methods related to specific items based on the user's cooking history. In this way, the assistant unit can select the optimal assistance method based on the user's past cooking history.

[0069] The assistant unit selects the optimal assistance method during assistance, taking into account the user's geographical location. For example, the assistant unit may provide assistance methods related to local specialties in the area where the user lives. For example, if the user is traveling, the assistant unit may provide assistance methods related to local cuisine in that area. For example, if the user is interested in a particular region, the assistant unit may also provide assistance methods related to the cuisine of that region. This allows the assistant unit to select the optimal assistance method based on the user's geographical location.

[0070] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0071] The data collection unit can analyze the user's eating history and determine the priority of information collection based on past eating patterns. For example, it can prioritize the collection of relevant information based on dishes the user has frequently eaten in the past. It can also filter information to be avoided based on dishes the user has avoided. It can also prioritize the collection of information about specific ingredients from the user's eating history. This allows the data collection unit to select the optimal information collection method based on the user's eating history.

[0072] The ordering department can analyze a user's past order history and select the optimal ordering method. For example, it can suggest relevant ingredients based on ingredients the user has frequently ordered in the past. It can also filter ingredients to avoid based on ingredients the user has avoided. It can also prioritize the collection of information about specific ingredients from the user's order history. As a result, the ordering department can select the optimal ordering method based on the user's past order history.

[0073] The improvement department can analyze past user feedback and select the most suitable improvement method. For example, it can propose relevant improvement methods based on feedback previously provided by users. It can also filter improvement methods to avoid based on those users have avoided. Furthermore, it can prioritize suggesting improvement methods for specific items based on the user's feedback history. In this way, the improvement department can select the most suitable improvement method based on past user feedback.

[0074] The assistant unit can analyze the user's past cooking history and select the most suitable assistance method. For example, it can suggest relevant assistance methods based on the user's past cooking experiences. It can also filter out assistance methods to avoid based on cooking methods the user has avoided. Furthermore, it can prioritize suggesting assistance methods related to specific items based on the user's cooking history. In this way, the assistant unit can select the most suitable assistance method based on the user's past cooking history.

[0075] The data collection unit can prioritize the collection of highly relevant information by considering the user's geographical location. For example, it can collect recipes using local specialties from the area where the user lives. If the user is traveling, it can collect recipes for local specialties from that region. If the user is interested in a particular region, it can also collect information about the cuisine of that region. In this way, the data collection unit can prioritize the collection of highly relevant information based on the user's geographical location.

[0076] The following briefly describes the processing flow for example form 1.

[0077] Step 1: The data collection unit gathers information about the user, such as their food preferences, allergies, health status, budget, and cooking skills. The data collection unit gathers information from user input and past data. For example, the user might provide information such as "I like spicy food," "I have a dairy allergy," "I'm trying to lose weight," "My budget is 5,000 yen per week," and "I'm a beginner cook." Step 2: The suggestion department proposes a week's worth of menus based on the information collected by the data collection department. The suggestion department uses AI to generate balanced menus that take into account the user's preferences, allergies, health condition, budget, and cooking skills. For example, it might suggest "Teriyaki Chicken and Stir-fried Vegetables" for Monday, "Grilled Fish and Salad" for Tuesday, and "Tofu Steak and Vegetable Soup" for Wednesday. Step 3: The ordering department generates an ingredient list based on the menu proposed by the suggestion department and places an automatic order. The ordering department generates an ingredient list necessary for the proposed menu and places an automatic order with a partner e-commerce site. For example, it generates an ingredient list such as "200g chicken," "1 cabbage," and "2 carrots," and places an order with the e-commerce site. Step 4: The serving department provides detailed recipes. The serving department provides step-by-step guides so that even beginner cooks can easily prepare the meals. For example, they suggest menus that minimize the number of utensils and cooking processes used, reducing the burden of cleanup. Step 5: The feedback section receives feedback from users. The feedback section allows users to leave ratings and comments on menus and recipes. For example, it receives feedback such as "This recipe was delicious" or "This menu is too time-consuming." Step 6: The Improvement Department improves the proposal based on the feedback received by the Feedback Department. The Improvement Department improves the quality of the proposal based on the feedback. For example, they incorporate the feedback into the next proposal. Step 7: The dialogue unit can adjust the menu with simple instructions (conversation). The dialogue unit can give simple instructions regarding the menu and recipes through voice input or chatbot-style conversation. For example, it can respond in real time to instructions such as, "I don't have much time today, so please change it to a menu that can be made quickly," or "I don't like spicy food, so please make a different menu." Step 8: The Assistant Department has an agent who acts as an assistant during cooking. The Assistant Department guides the cooking process step by step with voice and text, supporting even beginners so that they can cook without getting lost. For example, they will respond to instructions such as "Tell me the next step" or "Set the timer for 5 minutes."

[0078] (Example of form 2) The AI ​​agent system according to an embodiment of the present invention is a system that proposes a week's worth of menus, taking into account the user's food preferences, allergies, health status, budget, and cooking skills. This system collects information such as the user's food preferences, allergies, health status, budget, and cooking skills, and the AI ​​proposes a week's worth of menus based on the collected information. Furthermore, it generates a list of ingredients necessary for the proposed menu and automatically places orders with affiliated e-commerce sites. It also provides detailed recipes and suggestions that take cleanup into consideration. In addition, it is equipped with a function to suggest meal prepping for the weekend, a mechanism for receiving user feedback and an improvement cycle, an interactive function that allows menu adjustments with simple instructions (conversation), a function in which the agent acts as an assistant during cooking, autonomous ingredient management, health monitoring integration, smart kitchen integration, community planning, personalized nutrition guidance, social integration and sharing agent, and a world cuisine planner function. Furthermore, a "personal recipe advisor" function has been added. For example, it collects information such as the user's food preferences, allergies, health status, budget, and cooking skills. This information is either entered by the user or collected from past data. For example, a user provides information such as "I like spicy food," "I have a dairy allergy," "I'm trying to lose weight," "My budget is 5,000 yen per week," and "I'm a beginner cook." Next, based on the collected information, the AI ​​suggests a week's worth of meal plans. The AI ​​generates a balanced menu considering the user's preferences, allergies, health condition, budget, and cooking skills. For example, it might suggest "Teriyaki Chicken and Stir-fried Vegetables" for Monday, "Grilled Fish and Salad" for Tuesday, and "Tofu Steak and Vegetable Soup" for Wednesday. Furthermore, it generates a list of ingredients needed for the suggested menu and automatically places orders with partner e-commerce sites. This allows users to obtain the necessary ingredients without the hassle of purchasing them themselves. For example, an ingredient list such as "200g of chicken," "1 head of cabbage," and "2 carrots" is generated and ordered from the e-commerce site. The system also provides detailed recipes and suggestions that take cleanup into consideration. The AI ​​provides step-by-step guides so that even beginner cooks can easily prepare meals. It also suggests menus that minimize the use of equipment and cooking processes, reducing the burden of cleanup.For example, it suggests menus that can be made in one pan or require minimal washing up. It also has a feature that suggests meal prepping for the weekend. The AI ​​suggests menus that can be prepared in advance on the weekend, providing easy-to-eat meals for busy weekdays. For example, it might suggest "prepare curry on the weekend and reheat it on weekdays." It also has a system for receiving user feedback and an improvement cycle. Users can leave ratings and comments on menus and recipes, and the AI ​​uses this feedback to improve the quality of its suggestions. For example, it receives feedback such as "This recipe was delicious" or "This menu is too time-consuming" and incorporates it into future suggestions. It also has an interactive function that allows users to adjust menus with simple instructions (conversation). Users can give simple instructions to menus and recipes through voice input or chatbot-style conversation. For example, it responds in real time to instructions such as "I don't have much time today, so change it to a menu that can be made quickly" or "I don't like spicy food, so change it to a different menu." It also has a function where an agent acts as an assistant during cooking. The AI ​​guides the cooking process step by step with voice and text, supporting even beginners so that they can cook without getting lost. It also offers a timer function, visual support, question answering, and a voice assistant function. For example, it can respond to instructions such as "Tell me the next step" or "Set the timer to 5 minutes." It also features autonomous food management, health monitoring integration, smart kitchen integration, community planning, personalized nutrition guidance, social integration and sharing agent, and a World Cuisine Planner function. For example, it uses internet-connected sensors to automatically manage inventory in the refrigerator and pantry and automatically orders any missing ingredients from e-commerce sites. It also integrates with health data from wearable devices to adjust menus based on health status. Furthermore, it integrates with smart kitchen devices to autonomously control the cooking process. It also provides community-based meal planning, personalized nutrition guidance, social integration and sharing agent, and a World Cuisine Planner function. Finally, it adds a "Personal Recipe Advisor" function.The AI ​​learns the user's preferences and frustrations, and suggests personalized recipes. For example, it might suggest, "This user likes spicy food and is likely to like this combination of ingredients." It also accumulates user feedback to improve the accuracy of its suggestions. As a result, the AI ​​agent system can suggest a week's worth of meal plans, generate a list of necessary ingredients, and automatically place orders, taking into account the user's food preferences, allergies, health condition, budget, and cooking skills.

[0079] The AI ​​agent system according to this embodiment comprises a collection unit, a suggestion unit, an order unit, a delivery unit, a feedback unit, an improvement unit, a dialogue unit, and an assistant unit. The collection unit collects information such as the user's food preferences, allergies, health status, budget, and cooking skills. The collection unit collects information from, for example, information entered by the user and past data. For example, the collection unit receives information from the user such as "I like spicy food," "I have a dairy allergy," "I'm trying to lose weight," "My budget is 5,000 yen per week," and "I'm a beginner cook." The suggestion unit proposes a week's worth of menus based on the information collected by the collection unit. The suggestion unit uses, for example, AI to generate a balanced menu that takes into account the user's preferences, allergies, health status, budget, and cooking skills. For example, the suggestion unit proposes menus for each day, such as "Teriyaki chicken and stir-fried vegetables" on Monday, "Grilled fish and salad" on Tuesday, and "Tofu steak and vegetable soup" on Wednesday. The order unit generates a list of ingredients based on the menu proposed by the suggestion unit and places an automatic order. The ordering unit generates a list of ingredients needed for a suggested menu and automatically places orders with partner e-commerce sites. For example, the ordering unit generates an ingredient list such as "200g chicken," "1 cabbage," and "2 carrots" and places orders with e-commerce sites. The delivery unit provides detailed recipes. For example, the delivery unit provides step-by-step guides so that even beginner cooks can easily prepare the meals. For example, the delivery unit suggests menus that minimize the amount of equipment used and the cooking process, reducing the burden of cleanup. The feedback unit receives feedback from users. For example, the feedback unit allows users to leave ratings and comments on menus and recipes. For example, the feedback unit receives feedback such as "This recipe was delicious" or "This menu is too time-consuming." The improvement unit improves the suggestions based on the feedback received by the feedback unit. For example, the improvement unit improves the quality of suggestions based on the feedback. For example, the improvement unit incorporates the feedback into future suggestions. The dialogue unit allows users to adjust menus with simple instructions (conversation). The dialogue unit can, for example, give simple instructions regarding menus and recipes through voice input or a chatbot-style conversation.For example, the dialogue unit responds in real time to instructions such as, "I don't have much time today, so please change the menu to one that can be prepared quickly," or "I don't like spicy food, so please make a different menu." The assistant unit has an agent that acts as an assistant during cooking. The assistant unit, for example, guides the cooking process step by step with voice and text, supporting even beginners so that they can cook without getting lost. For example, the assistant unit responds to instructions such as, "Tell me the next step," or "Set the timer for 5 minutes." As a result, the AI ​​agent system according to this embodiment can propose a week's worth of menus, generate a list of necessary ingredients, and automatically order them, taking into account the user's food preferences, allergies, health condition, budget, cooking skills, etc.

[0080] The data collection unit collects information such as the user's food preferences, allergies, health status, budget, and cooking skills. For example, the unit collects information from user input and past data. Specifically, it collects information on the user's food preferences and allergies based on the information the user enters into the application. For instance, the user might provide information such as "I like spicy food," "I have a dairy allergy," "I'm trying to lose weight," "My budget is 5,000 yen per week," or "I'm a beginner cook." This information is saved as the user's profile and used for future suggestions and orders. The data collection unit also analyzes the user's past order history, ratings, and feedback to understand their preferences and tendencies. For example, it identifies the types of food and seasonings the user prefers based on ratings and comments on previously ordered dishes. Furthermore, the data collection unit also collects information on the user's health status. For example, if the user is linked to a health management app, the data is acquired to understand the user's health. This allows the data collection unit to suggest menus tailored to the user's health condition. The data collection unit centrally manages this information and provides it to the suggestion and ordering units. This allows the data collection unit to efficiently collect information that meets the diverse needs of users, thereby improving the overall accuracy and usability of the system.

[0081] The suggestion department proposes a week's worth of menus based on the information collected by the data collection department. The suggestion department uses AI, for example, to generate balanced menus that take into account the user's preferences, allergies, health condition, budget, and cooking skills. Specifically, the AI ​​analyzes the collected information and generates the optimal menu based on the user's preferences and constraints. For example, the suggestion department might suggest menus for each day, such as "Teriyaki Chicken and Stir-fried Vegetables" for Monday, "Grilled Fish and Salad" for Tuesday, and "Tofu Steak and Vegetable Soup" for Wednesday. The AI ​​also considers nutritional balance, calories, and cooking time when generating menus, allowing users to lead a healthy and efficient diet. Furthermore, the suggestion department improves the accuracy of its suggestions by incorporating user feedback. For example, if a user provides feedback such as "This recipe was delicious" or "This menu is too time-consuming," the suggestion department uses that information to improve its next suggestion. In this way, the suggestion department can provide the optimal menu that meets the user's needs and increase user satisfaction.

[0082] The ordering department generates ingredient lists and automatically places orders based on menus proposed by the suggestion department. For example, the ordering department generates an ingredient list necessary for a proposed menu and automatically places orders with partner e-commerce sites. Specifically, the ordering department lists the ingredients needed for each menu item in the proposed menu and places the optimal order considering the user's budget and inventory status. For example, the ordering department generates an ingredient list such as "200g chicken," "1 cabbage," and "2 carrots," and places an order with a partner e-commerce site. The ordering department manages the user's account information and delivery address information to ensure a smooth ordering process. The ordering department also monitors the order status and delivery status in real time and notifies the user. This allows the user to understand the progress of their order and receive ingredients with peace of mind. Furthermore, the ordering department can analyze the user's past order history and suggest repeat orders and subscriptions. In this way, the ordering department can improve user convenience and support efficient ingredient procurement.

[0083] The service provider offers detailed recipes. For example, they provide step-by-step guides so that even beginners can easily cook. Specifically, they suggest menus that minimize the amount of equipment and cooking process used, reducing the burden of cleanup. For instance, in the "Teriyaki Chicken" recipe, they provide detailed instructions from preparing the chicken to cooking and plating. Each step includes information on necessary equipment, cooking time, and precautions, allowing even beginners to cook without hesitation. The service provider also offers troubleshooting and advice during cooking. For example, they provide specific advice such as "reduce the heat if the chicken is about to burn" or "add water if the sauce is too thick." In this way, the service provider supports users so that they can cook with confidence and create delicious meals. Furthermore, the service provider also suggests variations and adaptations of recipes. For example, they offer methods for adapting "Teriyaki Chicken" into "Teriyaki Chicken Rice Bowl" or provide variations using different ingredients. In this way, the service provider helps users broaden their culinary horizons and increase their enjoyment of meals.

[0084] The Feedback Department receives feedback from users. For example, users can leave ratings and comments on menus and recipes. Specifically, the Feedback Department provides rating forms and comment sections within the application to make it easy for users to provide feedback. For example, users can enter ratings and comments such as "This recipe was delicious" or "This menu is too time-consuming." The Feedback Department collects this feedback and stores it in a database. The Feedback Department also analyzes user ratings and comments and provides them to the Suggestion Department and Improvement Department. This allows the Feedback Department to support service improvements that reflect user opinions. Furthermore, the Feedback Department can also provide replies and advice to user feedback. For example, it can reply with "I'm glad you liked this recipe" or "Please try this menu next time." This strengthens communication with users and increases user satisfaction.

[0085] The Improvement Department improves proposals based on feedback received by the Feedback Department. For example, the Improvement Department improves the quality of proposals based on feedback. Specifically, the Improvement Department analyzes user feedback and reflects it in the Proposal Department and the Delivery Department. For example, if a user provides feedback such as "This menu is too time-consuming," the Improvement Department will consider ways to simplify the cooking process and reflect this in the next proposal. The Improvement Department also understands user preferences and trends to provide more personalized proposals. For example, if a user provides feedback such as "I like spicy food," the Improvement Department will include more spicy dishes in the next proposal. In this way, the Improvement Department can provide optimal proposals that meet user needs and increase user satisfaction. Furthermore, the Improvement Department regularly reports the results of its feedback analysis and uses them to improve the entire system. In this way, the Improvement Department can achieve continuous service improvement and provide a more valuable system for users.

[0086] The dialogue unit can adjust menus with simple instructions (conversation). For example, the dialogue unit can give simple instructions regarding menus and recipes through voice input or chatbot-style dialogue. Specifically, when a user gives instructions by voice, such as "I don't have much time today, so change it to a menu that can be made quickly" or "I don't like spicy food, so please make a different menu," the AI ​​analyzes the instructions and adjusts the menu in real time. The dialogue unit uses natural language processing technology to accurately understand the user's instructions and respond appropriately. The dialogue unit also supports chatbot-style dialogue, allowing users to input instructions in text. For example, if a user inputs instructions such as "Review this week's menu" or "Suggest a new recipe," the dialogue unit will adjust the menu based on those instructions. This allows the dialogue unit to respond flexibly to the user's needs and improve user convenience. Furthermore, the dialogue unit saves the history of conversations with the user and uses it for future conversations. This allows the dialogue unit to understand the user's preferences and tendencies and provide more personalized responses.

[0087] The Assistant Unit acts as an assistant during cooking. For example, the Assistant Unit guides the cooking process step by step with voice and text, supporting even beginners so they can cook without getting lost. Specifically, when the user gives instructions such as "Tell me the next step" or "Set the timer for 5 minutes," the AI ​​provides appropriate guidance in response to those instructions. For example, it provides specific instructions in voice and text such as "Put the chicken in the frying pan and cook over medium heat for 5 minutes" and "Next, add the cabbage and sauté for another 3 minutes." The Assistant Unit also handles troubleshooting during cooking, providing appropriate advice to questions such as "What should I do if the sauce looks like it's going to burn?" In this way, the Assistant Unit supports users so they can cook with peace of mind. Furthermore, the Assistant Unit monitors the progress of cooking in real time and sets the timer or provides guidance on the next step as needed. In this way, the Assistant Unit can improve the user's cooking experience and support efficient and enjoyable cooking.

[0088] The collection unit is equipped with an autonomous food management agent. For example, the collection unit automatically manages the inventory of refrigerators and pantries using internet-connected sensors. For example, the collection unit automatically orders running low on ingredients from e-commerce sites. The collection unit can also manage the expiration dates of ingredients and suggest menus that prioritize the use of ingredients nearing their expiration date. This allows the collection unit to autonomously manage ingredients and automatically order running low on ingredients from e-commerce sites.

[0089] The data collection unit includes a health monitoring and menu adjustment agent. For example, the data collection unit works with health data from wearable devices to adjust menus based on the user's health status. For instance, the data collection unit suggests low-calorie or high-protein menus based on the user's health condition. It can also suggest nutritionally balanced menus based on the user's health goals. This allows the data collection unit to adjust menus based on the user's health status.

[0090] The data collection unit is equipped with a smart kitchen integration agent. The data collection unit can, for example, work with smart kitchen devices to autonomously control the cooking process. For example, the data collection unit can work with smart ovens and smart stoves to automatically set cooking temperatures and times. The data collection unit can also work with smart refrigerators to monitor food inventory in real time. As a result, the data collection unit can work with smart kitchen devices to autonomously control the cooking process.

[0091] The data collection unit includes a community-based meal planning agent. For example, the data collection unit collects and shares meal plans from communities in which users participate. For example, the data collection unit suggests new menus to users based on the meal plans of community members. The data collection unit can also collect popular recipes within a community and provide them to users. This enables the data collection unit to perform community-based meal planning.

[0092] The data collection unit is equipped with a personalized nutrition guidance agent. The data collection unit provides individualized nutrition guidance based, for example, the user's health status and food preferences. For instance, it suggests an appropriate nutritional balance according to the user's health goals. The data collection unit can also analyze the user's dietary history and evaluate their nutrient intake. This enables the data collection unit to provide personalized nutrition guidance.

[0093] The data collection unit includes social media integration and a sharing agent. For example, the data collection unit collects information about dishes shared by users on social media. For example, the data collection unit collects information based on the content of posts from cooking accounts that users follow. For example, the data collection unit can also analyze trends in cooking communities that users participate in and collect relevant information. This enables the data collection unit to perform social media integration and sharing.

[0094] The data collection unit includes a world cuisine planner function. For example, the data collection unit collects information about cuisines from around the world and provides it to the user. For example, the data collection unit suggests dishes from countries or regions that the user is interested in. The data collection unit can also provide recipes using ingredients from specific countries or regions. In this way, the data collection unit can provide a world cuisine planner function.

[0095] The data collection unit includes a personal recipe advisor. For example, the data collection unit learns the user's preferences and frustrations and suggests personalized recipes. For instance, it might suggest recipes based on the user's preference for spicy food and likely enjoyment of a particular ingredient combination. The data collection unit can also accumulate user feedback to improve the accuracy of its suggestions. This allows the data collection unit to provide a personalized recipe advisor function.

[0096] The data collection unit estimates the user's emotions and adjusts the timing of information collection based on the estimated emotions. For example, if the user is stressed, the data collection unit will collect information during relaxed times. For example, if the user is busy, the data collection unit will collect information during free time. For example, if the user is relaxed, the data collection unit can also collect information in real time. This allows the data collection unit to adjust the timing of information collection according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0097] The data collection unit analyzes the user's past eating history and selects the optimal information collection method. For example, the data collection unit collects relevant information based on the dishes the user has enjoyed eating in the past. For example, the data collection unit filters out information to be avoided based on the dishes the user has avoided in the past. For example, the data collection unit can also prioritize the collection of information about specific ingredients from the user's eating history. This allows the data collection unit to select the optimal information collection method based on the user's past eating history.

[0098] The data collection unit filters information based on the user's current health status and lifestyle. For example, if the user is on a diet, the unit prioritizes collecting low-calorie recipes. If the user has allergies, the unit collects recipes that do not contain allergens. If the user is busy, the unit can also collect recipes that can be prepared in a short amount of time. This allows the data collection unit to filter information based on the user's health status and lifestyle.

[0099] The data collection unit estimates the user's emotions and determines the priority of information to collect based on the estimated emotions. For example, if the user is stressed, the data collection unit will prioritize collecting recipes that promote relaxation. For example, if the user is relaxed, the data collection unit will prioritize collecting recipes that encourage trying new dishes. For example, if the user is busy, the data collection unit can also prioritize collecting recipes that are easy to make. In this way, the data collection unit can determine the priority of information to collect according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0100] The data collection unit prioritizes collecting highly relevant information, taking into account the user's geographical location. For example, it collects recipes using local specialties from the area where the user lives. If the user is traveling, it collects recipes for local specialties from that region. If the user is interested in a particular region, it can also collect information about the cuisine of that region. This allows the data collection unit to prioritize collecting highly relevant information based on the user's geographical location.

[0101] The data collection unit analyzes the user's social media activity and collects relevant information during data collection. For example, the data collection unit collects information about dishes that the user has shared on social media. For example, the data collection unit collects information based on the content of posts from cooking accounts that the user follows. For example, the data collection unit can also analyze trends in cooking communities that the user participates in and collect relevant information. In this way, the data collection unit can collect relevant information based on the user's social media activity.

[0102] The suggestion unit estimates the user's emotions and adjusts the way it presents suggestions based on those emotions. For example, if the user is stressed, the suggestion unit will present suggestions in a relaxing manner. If the user is relaxed, the suggestion unit will present suggestions in a more detailed manner. If the user is in a hurry, the suggestion unit can also present suggestions in a more concise manner. This allows the suggestion unit to adjust the way it presents suggestions according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may include, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0103] The suggestion function adjusts the level of detail in its suggestions based on the user's health condition. For example, it might suggest healthy recipes based on items identified in the user's health checkup. Or, if the user is on a diet, it might suggest recipes that take calories and nutritional balance into consideration. If the user has a specific illness, it might suggest meals suitable for that illness. This allows the suggestion function to adjust the level of detail in its suggestions based on the user's health condition.

[0104] The suggestion function applies different suggestion algorithms depending on the user's cooking skill level. For example, it suggests easy, foolproof recipes to beginners, slightly more elaborate recipes to intermediate cooks, and even challenging recipes to advanced cooks. This allows the suggestion function to apply different suggestion algorithms depending on the user's cooking skill level.

[0105] The suggestion unit estimates the user's emotions and adjusts the length of the suggestions based on the estimated emotions. For example, if the user is in a hurry, the suggestion unit will provide short, concise suggestions. If the user is relaxed, the suggestion unit will provide longer suggestions with detailed explanations. If the user is excited, the suggestion unit may also provide visually stimulating suggestions. This allows the suggestion unit to adjust the length of suggestions according to the user's emotions. Emotion estimation is achieved using emotion estimation functions, such as emotion engines or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0106] The suggestion function prioritizes suggestions based on the user's eating history. For example, it might prioritize suggesting dishes the user has enjoyed eating in the past. It might also exclude dishes the user has avoided in the past. The suggestion function can also prioritize suggesting recipes using specific ingredients based on the user's eating history. This allows the suggestion function to prioritize suggestions based on the user's eating history.

[0107] The suggestion function adjusts the order of suggestions based on the relevance of the user's meals. For example, it suggests recipes related to dishes the user has eaten in the past. For example, it suggests that the user avoid eating dishes containing certain ingredients consecutively. For example, it can suggest a balanced menu based on the user's meal history. This allows the suggestion function to adjust the order of suggestions based on the relevance of the user's meals.

[0108] The ordering system estimates the user's emotions and adjusts the timing of orders based on those emotions. For example, if the user is stressed, the ordering system will place an order during a relaxed time. If the user is busy, the ordering system will place an order during a free time. If the user is relaxed, the ordering system can even place an order in real time. This allows the ordering system to adjust the timing of orders according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0109] The ordering department analyzes the user's past order history to select the optimal ordering method when an order is placed. For example, the ordering department suggests relevant ingredients based on ingredients the user has frequently ordered in the past. For example, the ordering department filters out ingredients to avoid based on ingredients the user has avoided in the past. For example, the ordering department can also prioritize collecting information about specific ingredients from the user's order history. This allows the ordering department to select the optimal ordering method based on the user's past order history.

[0110] The ordering system customizes orders based on the user's current lifestyle. For example, if the user is busy, the system prioritizes ordering ingredients that are easy to cook. If the user is on a diet, the system prioritizes ordering low-calorie ingredients. If the user has allergies, the system can also order ingredients that do not contain allergens. This allows the ordering system to customize orders based on the user's current lifestyle.

[0111] The ordering system estimates the user's emotions and prioritizes orders based on those emotions. For example, if the user is stressed, the ordering system will prioritize ordering ingredients that promote relaxation. If the user is relaxed, the ordering system will prioritize ordering ingredients that allow them to try new dishes. If the user is busy, the ordering system may also prioritize ordering ingredients that are easy to prepare. This allows the ordering system to prioritize orders according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may include, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0112] The ordering system selects the optimal ordering method when an order is placed, taking into account the user's geographical location. For example, the ordering system might order ingredients that are specialties of the region where the user lives. If the user is traveling, the ordering system might order ingredients for local specialties of that region. If the user is interested in a particular region, the ordering system might also order ingredients related to the cuisine of that region. This allows the ordering system to select the optimal ordering method based on the user's geographical location.

[0113] The ordering department analyzes the user's social media activity when an order is placed and suggests the order. For example, the ordering department might order ingredients related to a dish the user has shared on social media. For example, the ordering department might order ingredients based on posts from cooking accounts the user follows. For example, the ordering department could also analyze trends in cooking communities the user participates in and order related ingredients. This allows the ordering department to suggest orders based on the user's social media activity.

[0114] The service provider estimates the user's emotions and adjusts the way the recipe is presented based on the estimated emotions. For example, if the user is stressed, the service provider will present the recipe in a relaxing manner. If the user is relaxed, the service provider will present the recipe in a manner that includes detailed information. If the user is in a hurry, the service provider can also present the recipe in a concise manner. This allows the service provider to adjust the way the recipe is presented according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may include, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0115] The service provider will provide detailed guides tailored to the user's cooking skill level when providing recipes. For example, they will provide step-by-step detailed guides for beginners, for example, guides that focus on the essentials for intermediate cooks, and guides that include suggestions for variations for advanced cooks. This allows the service provider to provide detailed guides tailored to the user's cooking skill level.

[0116] The service provider customizes recipes based on the user's health condition when providing them. For example, if a user is on a diet, the service provider will provide low-calorie recipes. If a user has a specific illness, the service provider will provide recipes suitable for that illness. The service provider can also provide healthy recipes based on items pointed out to the user during a health checkup. In this way, the service provider can customize recipes based on the user's health condition.

[0117] The service provider estimates the user's emotions and prioritizes recipes based on those emotions. For example, if the user is stressed, the service provider will prioritize recipes that promote relaxation. If the user is relaxed, the service provider will prioritize recipes that encourage them to try new dishes. If the user is busy, the service provider can also prioritize easy-to-make recipes. This allows the service provider to prioritize recipes according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may include, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0118] The service provider adjusts recipe content based on the user's eating history when providing recipes. For example, the service provider provides relevant recipes based on dishes the user has enjoyed eating in the past. For example, the service provider filters out recipes to avoid based on dishes the user has avoided in the past. For example, the service provider can also prioritize providing recipes related to specific ingredients based on the user's eating history. In this way, the service provider can adjust recipe content based on the user's eating history.

[0119] The service provider analyzes the user's social media activity when providing recipes and suggests recipe content accordingly. For example, the service provider may provide recipes related to dishes the user has shared on social media. For example, the service provider may provide recipes based on posts from cooking accounts the user follows. For example, the service provider may analyze trends in cooking communities the user participates in and provide relevant recipes. This allows the service provider to suggest recipe content based on the user's social media activity.

[0120] The feedback unit estimates the user's emotions and adjusts the feedback collection method based on the estimated emotions. For example, if the user is stressed, the feedback unit collects feedback in the form of simple questions. For example, if the user is relaxed, the feedback unit requests detailed feedback. For example, if the user is in a hurry, the feedback unit can also provide a feedback format that can be answered quickly. This allows the feedback unit to adjust the feedback collection method according to the user's emotions. Emotion estimation is achieved using emotion estimation functions, such as emotion engines or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0121] The feedback unit analyzes the user's past feedback history to select the optimal collection method when collecting feedback. For example, the feedback unit suggests relevant questions based on the feedback the user has provided in the past. For example, the feedback unit filters out questions to avoid based on the question formats the user has avoided in the past. For example, the feedback unit can also prioritize collecting feedback on specific items from the user's feedback history. This allows the feedback unit to select the optimal collection method based on the user's past feedback history.

[0122] The feedback unit estimates the user's emotions and prioritizes feedback based on the estimated emotions. For example, if the user is stressed, the feedback unit will prioritize collecting relaxing feedback. For example, if the user is relaxed, the feedback unit will prioritize collecting feedback on new suggestions. For example, if the user is busy, the feedback unit may also prioritize collecting feedback that can be easily answered. In this way, the feedback unit can prioritize feedback according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0123] The feedback unit selects the optimal collection method when collecting feedback, taking into account the user's geographical location. For example, the feedback unit collects feedback on local specialties in the area where the user lives. For example, if the user is traveling, the feedback unit collects feedback on local cuisine in that area. For example, if the user is interested in a particular region, the feedback unit can also collect feedback on the cuisine of that region. This allows the feedback unit to select the optimal collection method based on the user's geographical location.

[0124] The improvement unit estimates the user's emotions and adjusts the improvement method based on the estimated emotions. For example, if the user is stressed, the improvement unit suggests a relaxing improvement method. For example, if the user is relaxed, the improvement unit suggests a detailed improvement method. For example, if the user is in a hurry, the improvement unit can also suggest a concise improvement method. In this way, the improvement unit can adjust the improvement method according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0125] The improvement department analyzes past user feedback to select the optimal improvement method during the improvement process. For example, the improvement department proposes relevant improvement methods based on feedback previously provided by users. For example, the improvement department filters out improvement methods to be avoided based on improvement methods that users have avoided in the past. For example, the improvement department can also prioritize suggesting improvement methods for specific items based on the user's feedback history. This allows the improvement department to select the optimal improvement method based on past user feedback.

[0126] The improvement unit estimates the user's emotions and determines the priority of improvements based on the estimated emotions. For example, if the user is stressed, the improvement unit will prioritize suggesting improvements that promote relaxation. For example, if the user is relaxed, the improvement unit will prioritize suggesting improvements related to new ideas. For example, if the user is busy, the improvement unit can also prioritize suggesting improvements that are easy to implement. In this way, the improvement unit can determine the priority of improvements according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0127] The improvement unit selects the optimal improvement method when making improvements, taking into account the user's geographical location information. For example, the improvement unit may suggest improvement methods related to local specialties in the area where the user lives. For example, if the user is traveling, the improvement unit may suggest improvement methods related to local cuisine in that area. For example, if the user is interested in a particular region, the improvement unit may also suggest improvement methods related to the cuisine of that region. In this way, the improvement unit can select the optimal improvement method based on the user's geographical location information.

[0128] The dialogue unit estimates the user's emotions and adjusts the dialogue method based on the estimated emotions. For example, if the user is stressed, the dialogue unit provides a relaxing dialogue method. For example, if the user is relaxed, the dialogue unit provides a dialogue method that includes detailed information. For example, if the user is in a hurry, the dialogue unit can also provide a concise dialogue method. In this way, the dialogue unit can adjust the dialogue method according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0129] The dialogue unit analyzes the user's past dialogue history to select the optimal dialogue method during a conversation. For example, the dialogue unit suggests relevant dialogue methods based on the user's past conversations. For example, the dialogue unit filters out dialogue methods to be avoided based on dialogue methods the user has avoided in the past. For example, the dialogue unit can also prioritize suggesting dialogue methods related to specific items based on the user's dialogue history. In this way, the dialogue unit can select the optimal dialogue method based on the user's past dialogue history.

[0130] The dialogue unit estimates the user's emotions and prioritizes dialogue based on the estimated emotions. For example, if the user is stressed, the dialogue unit will prioritize providing relaxing dialogue. For example, if the user is relaxed, the dialogue unit will prioritize providing dialogue about new suggestions. For example, if the user is busy, the dialogue unit can also prioritize providing dialogue that can be easily performed. In this way, the dialogue unit can determine the priority of dialogue according to the user's emotions. Emotion estimation is achieved using emotion estimation functions, such as emotion engines or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0131] The dialogue unit selects the optimal dialogue method during a conversation, taking into account the user's geographical location. For example, the dialogue unit may provide a dialogue method about local specialties in the area where the user lives. For example, if the user is traveling, the dialogue unit may provide a dialogue method about local cuisine in that area. For example, if the user is interested in a particular region, the dialogue unit may also provide a dialogue method about the cuisine of that region. In this way, the dialogue unit can select the optimal dialogue method based on the user's geographical location.

[0132] The assistant unit estimates the user's emotions and adjusts its methods based on the estimated emotions. For example, if the user is stressed, the assistant unit provides a relaxing method. If the user is relaxed, the assistant unit provides a method that includes detailed information. If the user is in a hurry, the assistant unit can also provide a concise method. This allows the assistant unit to adjust its methods according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0133] The assistant unit analyzes the user's past cooking history to select the optimal assistance method during assistance. For example, the assistant unit suggests relevant assistance methods based on the user's past cooking experiences. For example, the assistant unit filters out assistance methods to be avoided based on cooking methods the user has avoided in the past. For example, the assistant unit can also prioritize suggesting assistance methods related to specific items based on the user's cooking history. In this way, the assistant unit can select the optimal assistance method based on the user's past cooking history.

[0134] The assistant unit estimates the user's emotions and prioritizes assistants based on the estimated emotions. For example, if the user is stressed, the assistant unit will prioritize providing a relaxing assistant. If the user is relaxed, the assistant unit will prioritize providing an assistant with new suggestions. If the user is busy, the assistant unit can also prioritize providing an assistant that can be easily performed. In this way, the assistant unit can prioritize assistants according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0135] The assistant unit selects the optimal assistance method during assistance, taking into account the user's geographical location. For example, the assistant unit may provide assistance methods related to local specialties in the area where the user lives. For example, if the user is traveling, the assistant unit may provide assistance methods related to local cuisine in that area. For example, if the user is interested in a particular region, the assistant unit may also provide assistance methods related to the cuisine of that region. This allows the assistant unit to select the optimal assistance method based on the user's geographical location.

[0136] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0137] The suggestion function can estimate the user's emotions and adjust the content of its suggestions based on those emotions. For example, if the user is feeling stressed, it can suggest a menu that promotes relaxation. If the user is relaxed, it can suggest a menu that encourages them to try a new dish. If the user is busy, it can suggest a menu that can be prepared quickly. In this way, the suggestion function can adjust the content of its suggestions according to the user's emotions.

[0138] The data collection unit can analyze the user's eating history and determine the priority of information collection based on past eating patterns. For example, it can prioritize the collection of relevant information based on dishes the user has frequently eaten in the past. It can also filter information to be avoided based on dishes the user has avoided. It can also prioritize the collection of information about specific ingredients from the user's eating history. This allows the data collection unit to select the optimal information collection method based on the user's eating history.

[0139] The service provider can estimate the user's emotions and adjust the way the recipe is presented based on those emotions. For example, if the user is stressed, the recipe can be presented in a relaxing manner. If the user is relaxed, the recipe can be presented in a way that includes detailed information. If the user is in a hurry, the recipe can be presented in a concise manner. In this way, the service provider can adjust the way the recipe is presented according to the user's emotions.

[0140] The ordering department can analyze a user's past order history and select the optimal ordering method. For example, it can suggest relevant ingredients based on ingredients the user has frequently ordered in the past. It can also filter ingredients to avoid based on ingredients the user has avoided. It can also prioritize the collection of information about specific ingredients from the user's order history. As a result, the ordering department can select the optimal ordering method based on the user's past order history.

[0141] The feedback unit can estimate the user's emotions and adjust the feedback collection method based on the estimated emotions. For example, if the user is stressed, feedback can be collected in the form of simple questions. If the user is relaxed, detailed feedback can be requested. If the user is in a hurry, a feedback format that can be answered quickly can be provided. In this way, the feedback unit can adjust the feedback collection method according to the user's emotions.

[0142] The improvement department can analyze past user feedback and select the most suitable improvement method. For example, it can propose relevant improvement methods based on feedback previously provided by users. It can also filter improvement methods to avoid based on those users have avoided. Furthermore, it can prioritize suggesting improvement methods for specific items based on the user's feedback history. In this way, the improvement department can select the most suitable improvement method based on past user feedback.

[0143] The dialogue unit can estimate the user's emotions and adjust the dialogue method based on those emotions. For example, if the user is stressed, it can provide a relaxing dialogue method. If the user is relaxed, it can provide a dialogue method that includes detailed information. If the user is in a hurry, it can provide a concise dialogue method. In this way, the dialogue unit can adjust the dialogue method according to the user's emotions.

[0144] The assistant unit can analyze the user's past cooking history and select the most suitable assistance method. For example, it can suggest relevant assistance methods based on the user's past cooking experiences. It can also filter out assistance methods to avoid based on cooking methods the user has avoided. Furthermore, it can prioritize suggesting assistance methods related to specific items based on the user's cooking history. In this way, the assistant unit can select the most suitable assistance method based on the user's past cooking history.

[0145] The data collection unit can prioritize the collection of highly relevant information by considering the user's geographical location. For example, it can collect recipes using local specialties from the area where the user lives. If the user is traveling, it can collect recipes for local specialties from that region. If the user is interested in a particular region, it can also collect information about the cuisine of that region. In this way, the data collection unit can prioritize the collection of highly relevant information based on the user's geographical location.

[0146] The suggestion function can estimate the user's emotions and adjust the way suggestions are presented based on those emotions. For example, if the user is stressed, the suggestion can be presented in a relaxing manner. If the user is relaxed, the suggestion can be presented in a way that includes detailed information. If the user is in a hurry, the suggestion can be presented in a concise manner. In this way, the suggestion function can adjust the way suggestions are presented according to the user's emotions.

[0147] The following briefly describes the processing flow for example form 2.

[0148] Step 1: The data collection unit gathers information about the user, such as their food preferences, allergies, health status, budget, and cooking skills. The data collection unit gathers information from user input and past data. For example, the user might provide information such as "I like spicy food," "I have a dairy allergy," "I'm trying to lose weight," "My budget is 5,000 yen per week," and "I'm a beginner cook." Step 2: The suggestion department proposes a week's worth of menus based on the information collected by the data collection department. The suggestion department uses AI to generate balanced menus that take into account the user's preferences, allergies, health condition, budget, and cooking skills. For example, it might suggest "Teriyaki Chicken and Stir-fried Vegetables" for Monday, "Grilled Fish and Salad" for Tuesday, and "Tofu Steak and Vegetable Soup" for Wednesday. Step 3: The ordering department generates an ingredient list based on the menu proposed by the suggestion department and places an automatic order. The ordering department generates an ingredient list necessary for the proposed menu and places an automatic order with a partner e-commerce site. For example, it generates an ingredient list such as "200g chicken," "1 cabbage," and "2 carrots," and places an order with the e-commerce site. Step 4: The serving department provides detailed recipes. The serving department provides step-by-step guides so that even beginner cooks can easily prepare the meals. For example, they suggest menus that minimize the number of utensils and cooking processes used, reducing the burden of cleanup. Step 5: The feedback section receives feedback from users. The feedback section allows users to leave ratings and comments on menus and recipes. For example, it receives feedback such as "This recipe was delicious" or "This menu is too time-consuming." Step 6: The Improvement Department improves the proposal based on the feedback received by the Feedback Department. The Improvement Department improves the quality of the proposal based on the feedback. For example, they incorporate the feedback into the next proposal. Step 7: The dialogue unit can adjust the menu with simple instructions (conversation). The dialogue unit can give simple instructions regarding the menu and recipes through voice input or chatbot-style conversation. For example, it can respond in real time to instructions such as, "I don't have much time today, so please change it to a menu that can be made quickly," or "I don't like spicy food, so please make a different menu." Step 8: The Assistant Department has an agent who acts as an assistant during cooking. The Assistant Department guides the cooking process step by step with voice and text, supporting even beginners so that they can cook without getting lost. For example, they will respond to instructions such as "Tell me the next step" or "Set the timer for 5 minutes."

[0149] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0150] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include text generation AI, image generation AI, and multimodal generation AI. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats from audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each of the above parts is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example.Furthermore, processing performed by AI, including generative AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generative AI.

[0151] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0152] Each of the multiple elements described above, including the collection unit, proposal unit, order unit, delivery unit, feedback unit, improvement unit, dialogue unit, and assistant unit, is implemented, for example, by at least one of the smart device 14 and the data processing unit 12. For example, the collection unit uses the camera 42 and microphone 38B of the smart device 14 to collect the user's food preferences and allergy information and transmits it to the data processing unit 12 via the control unit 46A. The proposal unit is implemented, for example, by the specific processing unit 290 of the data processing unit 12 and generates a week's worth of menus based on the collected information. The order unit is implemented, for example, by the specific processing unit 290 of the data processing unit 12 and generates a list of ingredients based on the proposed menu and automatically places an order with a partner e-commerce site. The delivery unit is implemented, for example, by the control unit 46A of the smart device 14 and provides the user with detailed recipes. The feedback unit is implemented, for example, by the control unit 46A of the smart device 14 and collects feedback from the user. The improvement unit is implemented, for example, by the specific processing unit 290 of the data processing unit 12 and improves the proposals based on the feedback. The dialogue unit is implemented, for example, by the control unit 46A of the smart device 14, and handles conversations via voice input or chatbot format. The assistant unit is implemented, for example, by the control unit 46A of the smart device 14, and an agent acts as an assistant during cooking. The correspondence between each unit and the device or control unit is not limited to the examples described above and can be modified in various ways.

[0153] [Second Embodiment] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0154] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0155] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0156] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0157] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0158] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0159] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0160] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing by the processor 28. The storage 32 stores the specific processing program 56.

[0161] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0162] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0163] In the smart glasses 214, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0164] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0165] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0166] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0167] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0168] Each of the multiple elements described above, including the collection unit, suggestion unit, order unit, provision unit, feedback unit, improvement unit, dialogue unit, and assistant unit, is implemented, for example, in at least one of the smart glasses 214 and the data processing unit 12. For example, the collection unit uses the camera 42 and microphone 238 of the smart glasses 214 to collect the user's food preferences and allergy information and transmits it to the data processing unit 12 via the control unit 46A. The suggestion unit is implemented, for example, by the identification processing unit 290 of the data processing unit 12 and generates a week's worth of menus based on the collected information. The order unit is implemented, for example, by the identification processing unit 290 of the data processing unit 12 and generates a list of ingredients based on the suggested menus and automatically places orders with affiliated e-commerce sites. The provision unit is implemented, for example, by the control unit 46A of the smart glasses 214 and provides the user with detailed recipes. The feedback unit is implemented, for example, by the control unit 46A of the smart glasses 214 and collects feedback from the user. The improvement unit is implemented, for example, by the specific processing unit 290 of the data processing device 12, and improves the suggestions based on feedback. The dialogue unit is implemented, for example, by the control unit 46A of the smart glasses 214, and engages in dialogue via voice input or chatbot format. The assistant unit is implemented, for example, by the control unit 46A of the smart glasses 214, and an agent acts as an assistant during cooking. The correspondence between each unit and the devices and control units is not limited to the examples described above, and various modifications are possible.

[0169] [Third Embodiment] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0170] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0171] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0172] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0173] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0174] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0175] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0176] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0177] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0178] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0179] In the headset terminal 314, specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50 and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific program 60 executed on the RAM 48. The headset terminal 314 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0180] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0181] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0182] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0183] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0184] Each of the multiple elements described above, including the collection unit, suggestion unit, order unit, provision unit, feedback unit, improvement unit, dialogue unit, and assistant unit, is implemented by, for example, at least one of the headset terminal 314 and the data processing unit 12. For example, the collection unit uses the camera 42 and microphone 238 of the headset terminal 314 to collect the user's food preferences and allergy information and transmits it to the data processing unit 12 via the control unit 46A. The suggestion unit is implemented, for example, by the identification processing unit 290 of the data processing unit 12 and generates a week's worth of menus based on the collected information. The order unit is implemented, for example, by the identification processing unit 290 of the data processing unit 12 and generates a list of ingredients based on the suggested menus and automatically places orders with affiliated e-commerce sites. The provision unit is implemented, for example, by the control unit 46A of the headset terminal 314 and provides the user with detailed recipes. The feedback unit is implemented, for example, by the control unit 46A of the headset terminal 314 and collects feedback from the user. The improvement unit is implemented, for example, by the specific processing unit 290 of the data processing device 12, and improves the suggestions based on feedback. The dialogue unit is implemented, for example, by the control unit 46A of the headset terminal 314, and performs dialogue via voice input or chatbot format. The assistant unit is implemented, for example, by the control unit 46A of the headset terminal 314, and an agent acts as an assistant during cooking. The correspondence between each unit and the devices and control units is not limited to the examples described above, and various changes are possible.

[0185] [Fourth Embodiment] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0186] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0187] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0188] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0189] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0190] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS image sensor or CCD image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0191] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0192] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. The robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0193] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0194] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0195] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0196] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in storage 50. Processor 46 reads the specific program 60 from storage 50 and executes it on RAM 48. The specific processing is achieved by processor 46 acting as a control unit 46A according to the specific program 60 executed on RAM 48. Robot 414 also has data generation model 58 and emotion identification model 59, similar to those of the robot, and can perform processing similar to that of the specific processing unit 290 using these models.

[0197] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0198] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0199] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0200] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the robot 414 or an external device, and the robot 414 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0201] Each of the multiple elements described above, including the collection unit, proposal unit, order unit, delivery unit, feedback unit, improvement unit, dialogue unit, and assistant unit, is implemented by, for example, at least one of the robot 414 and the data processing unit 12. For example, the collection unit uses the camera 42 and microphone 238 of the robot 414 to collect the user's food preferences and allergy information and transmits it to the data processing unit 12 via the control unit 46A. The proposal unit is implemented, for example, by the specific processing unit 290 of the data processing unit 12 and generates a week's worth of menus based on the collected information. The order unit is implemented, for example, by the specific processing unit 290 of the data processing unit 12 and generates a list of ingredients based on the proposed menu and automatically places an order with a partner e-commerce site. The delivery unit is implemented, for example, by the control unit 46A of the robot 414 and provides the user with detailed recipes. The feedback unit is implemented, for example, by the control unit 46A of the robot 414 and collects feedback from the user. The improvement unit is implemented, for example, by the specific processing unit 290 of the data processing unit 12 and improves the proposals based on the feedback. The dialogue unit is implemented, for example, by the control unit 46A of the robot 414, and handles dialogue via voice input or chatbot format. The assistant unit is implemented, for example, by the control unit 46A of the robot 414, and an agent acts as an assistant during cooking. The correspondence between each unit and the device or control unit is not limited to the examples described above and can be modified in various ways.

[0202] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0203] Figure 9 shows the emotion map 400, in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0204] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0205] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0206] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, and motorcycles, emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0207] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0208] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0209] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0210] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0211] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0212] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0213] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0214] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0215] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0216] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0217] Furthermore, although the above-described examples were divided into four embodiments, some or all of these embodiments may be combined. Also, the smart device 14, smart glasses 214, headset terminal 314, and robot 414 are just examples, and they may be combined, or other devices may be used. Also, although the above-described examples were divided into two embodiments, Embodiment 1 and Embodiment 2, these may be combined.

[0218] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and other things that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0219] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0220] (Note 1) A data collection unit that collects information such as the user's food preferences, allergies, health status, budget, and cooking skills, Based on the information collected by the aforementioned collection unit, a proposal unit proposes a menu for one week, An ordering unit generates a list of ingredients based on the menu proposed by the aforementioned proposal unit and automatically places an order. A department that provides detailed recipes, A feedback unit that receives feedback from users, An improvement unit that improves the proposal based on the feedback received by the aforementioned feedback unit, A dialogue unit that allows you to adjust the menu with simple instructions, It includes an assistant department where agents act as assistants during cooking. A system characterized by the following features. (Note 2) The aforementioned collection unit is Equipped with an autonomous food management agent The system described in Appendix 1, characterized by the features described herein. (Note 3) The aforementioned collection unit is Equipped with health monitoring and menu adjustment agents. The system described in Appendix 1, characterized by the features described herein. (Note 4) The aforementioned collection unit is Equipped with a smart kitchen integration agent The system described in Appendix 1, characterized by the features described herein. (Note 5) The aforementioned collection unit is Features a community-based meal planning agent. The system described in Appendix 1, characterized by the features described herein. (Note 6) The aforementioned collection unit is Equipped with a personalized nutrition counseling agent The system described in Appendix 1, characterized by the features described herein. (Note 7) The aforementioned collection unit is Equipped with social networking and a sharing agent. The system described in Appendix 1, characterized by the features described herein. (Note 8) The aforementioned collection unit is Equipped with a World Cuisine Planner function The system described in Appendix 1, characterized by the features described herein. (Note 9) The aforementioned collection unit is Includes a personal recipe advisor. The system described in Appendix 1, characterized by the features described herein. (Note 10) The aforementioned collection unit is It estimates the user's emotions and adjusts the timing of information collection based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 11) The aforementioned collection unit is Analyze the user's past meal history and select the optimal method for collecting information. The system described in Appendix 1, characterized by the features described herein. (Note 12) The aforementioned collection unit is When collecting information, filtering is performed based on the user's current health status and lifestyle. The system described in Appendix 1, characterized by the features described herein. (Note 13) The aforementioned collection unit is It estimates the user's emotions and prioritizes the information to collect based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 14) The aforementioned collection unit is When collecting information, the system prioritizes collecting highly relevant information by considering the user's geographical location. The system described in Appendix 1, characterized by the features described herein. (Note 15) The aforementioned collection unit is When gathering information, we analyze users' social media activity and collect relevant information. The system described in Appendix 1, characterized by the features described herein. (Note 16) The aforementioned proposal section is, It estimates the user's emotions and adjusts the way suggestions are presented based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 17) The aforementioned proposal section is, When making a proposal, adjust the level of detail based on the user's health status. The system described in Appendix 1, characterized by the features described herein. (Note 18) The aforementioned proposal section is, When making suggestions, different suggestion algorithms are applied depending on the user's cooking skill level. The system described in Appendix 1, characterized by the features described herein. (Note 19) The aforementioned proposal section is, It estimates the user's emotions and adjusts the length of the suggestion based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 20) The aforementioned proposal section is, When making suggestions, the priority of suggestions is determined based on the user's meal history. The system described in Appendix 1, characterized by the features described herein. (Note 21) The aforementioned proposal section is, When making suggestions, the order of suggestions is adjusted based on the relevance of the user's meals. The system described in Appendix 1, characterized by the features described herein. (Note 22) The aforementioned ordering section is, It estimates the user's emotions and adjusts the timing of orders based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 23) The aforementioned ordering section is, When an order is placed, the system analyzes the user's past order history to select the optimal ordering method. The system described in Appendix 1, characterized by the features described herein. (Note 24) The aforementioned ordering section is, When placing an order, the order contents are customized based on the user's current living situation. The system described in Appendix 1, characterized by the features described herein. (Note 25) The aforementioned ordering section is, It estimates the user's emotions and determines order priorities based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 26) The aforementioned ordering section is, When an order is placed, the system selects the optimal ordering method by considering the user's geographical location. The system described in Appendix 1, characterized by the features described herein. (Note 27) The aforementioned ordering section is, When you place an order, we analyze your social media activity and suggest suitable order options. The system described in Appendix 1, characterized by the features described herein. (Note 28) The aforementioned supply unit is, The system estimates the user's emotions and adjusts the recipe delivery method based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 29) The aforementioned supply unit is, When providing recipes, we offer detailed guidance tailored to the user's cooking skill level. The system described in Appendix 1, characterized by the features described herein. (Note 30) The aforementioned supply unit is, When providing recipes, customize the recipe content based on the user's health condition. The system described in Appendix 1, characterized by the features described herein. (Note 31) The aforementioned supply unit is, It estimates the user's emotions and prioritizes recipes based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 32) The aforementioned supply unit is, When providing recipes, the recipe content is adjusted based on the user's eating history. The system described in Appendix 1, characterized by the features described herein. (Note 33) The aforementioned supply unit is, When providing recipes, we analyze users' social media activity to suggest recipe content. The system described in Appendix 1, characterized by the features described herein. (Note 34) The aforementioned feedback unit is We estimate the user's emotions and adjust the feedback collection method based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 35) The aforementioned feedback unit is When collecting feedback, the system analyzes the user's past feedback history to select the optimal collection method. The system described in Appendix 1, characterized by the features described herein. (Note 36) The aforementioned feedback unit is It estimates the user's emotions and prioritizes feedback based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 37) The aforementioned feedback unit is When collecting feedback, the optimal collection method is selected considering the user's geographical location. The system described in Appendix 1, characterized by the features described herein. (Note 38) The aforementioned improvement unit is, It estimates user sentiment and adjusts improvement methods based on the estimated user sentiment. The system described in Appendix 1, characterized by the features described herein. (Note 39) The aforementioned improvement unit is, When making improvements, we analyze past user feedback to select the most suitable improvement method. The system described in Appendix 1, characterized by the features described herein. (Note 40) The aforementioned improvement unit is, We estimate user emotions and determine improvement priorities based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Appendix 41) The improvement unit selects an optimal improvement method considering the user's geographical location information during improvement. The system according to Appendix 1, characterized in that. (Appendix 42) The dialogue unit estimates the user's emotion and adjusts the dialogue method based on the estimated user's emotion. The system according to Appendix 1, characterized in that. (Appendix 43) The dialogue unit selects an optimal dialogue method by analyzing the user's past dialogue history during dialogue. The system according to Appendix 1, characterized in that. (Appendix 44) The dialogue unit estimates the user's emotion and determines the dialogue priority based on the estimated user's emotion. The system according to Appendix 1, characterized in that. (Appendix 45) The dialogue unit selects an optimal dialogue method considering the user's geographical location information during dialogue. The system according to Appendix 1, characterized in that. (Appendix 46) The assistant unit estimates the user's emotion and adjusts the assistant method based on the estimated user's emotion. The system according to Appendix 1, characterized in that. (Appendix 47) The assistant unit [[ID=�3]]selects an optimal assistant method by analyzing the user's past cooking history during assistant. The system according to Appendix 1, characterized in that. (Appendix 48) The assistant unit estimates the user's emotion and determines the assistant priority based on the estimated user's emotion. The system according to Appendix 1, characterized in that. (Note 49) The aforementioned assistant section is During assistance, the system selects the optimal assistance method by considering the user's geographical location. The system described in Appendix 1, characterized by the features described herein. [Explanation of symbols]

[0221] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots

Claims

1. A data collection unit that collects information such as the user's food preferences, allergies, health status, budget, and cooking skills, Based on the information collected by the aforementioned collection unit, a proposal unit proposes a menu for one week, An ordering unit generates a list of ingredients based on the menu proposed by the aforementioned proposal unit and automatically places an order. A department that provides detailed recipes, A feedback unit that receives feedback from users, An improvement unit that improves the proposal based on the feedback received by the aforementioned feedback unit, A dialogue unit that allows you to adjust the menu with simple instructions, It includes an assistant department where agents act as assistants during cooking. A system characterized by the following features.

2. The aforementioned collection unit is Equipped with an autonomous food management agent The system according to feature 1.

3. The aforementioned collection unit is Equipped with health monitoring and menu adjustment agents. The system according to feature 1.

4. The aforementioned collection unit is Equipped with a smart kitchen integration agent The system according to feature 1.

5. The aforementioned collection unit is Features a community-based meal planning agent. The system according to feature 1.

6. The aforementioned collection unit is Equipped with a personalized nutrition counseling agent The system according to feature 1.

7. The aforementioned collection unit is Equipped with social networking and a sharing agent. The system according to feature 1.

8. The aforementioned collection unit is Equipped with a World Cuisine Planner function The system according to feature 1.

Images