system

Abstract

The system according to this embodiment aims to provide real-time advice during the cooking process, enabling the user to create a satisfying meal. [Solution] The system according to the embodiment comprises an analysis unit, a suggestion unit, a monitoring unit, and a feedback unit. The analysis unit analyzes the inventory and purchase history of ingredients. The suggestion unit proposes the optimal recipe based on the information analyzed by the analysis unit. The monitoring unit monitors the cooking process in real time based on the recipe proposed by the suggestion unit and provides advice at the appropriate time. The feedback unit provides a successful experience with a customized recipe based on the information monitored by the monitoring unit and accumulates feedback for future use.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology, there is a problem that real-time advice cannot be obtained during the cooking process, and it is difficult for the user to cook a satisfactory dish.

[0005] The system according to the embodiment aims to provide real-time advice during the cooking process and enable the user to cook a satisfactory dish.

Means for Solving the Problems

[0006] The system according to this embodiment comprises an analysis unit, a suggestion unit, a monitoring unit, and a feedback unit. The analysis unit analyzes the inventory and purchase history of ingredients. The suggestion unit proposes an optimal recipe based on the information analyzed by the analysis unit. The monitoring unit monitors the cooking process in real time based on the recipe proposed by the suggestion unit and provides advice at the appropriate time. The feedback unit provides a successful experience using a customized recipe based on the information monitored by the monitoring unit and accumulates feedback for future use. [Effects of the Invention]

[0007] The system according to this embodiment provides real-time advice during the cooking process, enabling the user to create a satisfying dish. [Brief explanation of the drawing]

[0008] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9]This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Modes for carrying out the invention]

[0009] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0010] First, let's explain the terminology used in the following explanation.

[0011] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit).

[0012] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0013] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0014] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor, an antenna, and the like. The communication I / F manages communication between multiple computers. Examples of communication standards applicable to the communication I / F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0016] [First Embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. Also, the database 24 and the communication I / F 26 are connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0020] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, and accepts user input. The touch panel 38A accepts user input via touch by detecting contact with an object (e.g., a pen or finger). The microphone 38B accepts user input via voice by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 (see Figure 2) acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user by outputting the data in a form perceptible to the user (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0023] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0025] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0026] In the smart device 14, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0027] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device having the data generation model 58. The data processing device 12 may also be a server device or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example of form 1) The cooking support system according to an embodiment of the present invention is a system that uses an AI agent to maximize the enjoyment of cooking and improve cooking skills. When a user starts cooking, the AI ​​agent automatically analyzes the user's ingredient inventory and purchase history. Next, the AI ​​agent suggests an optimal recipe based on the user's taste preferences and ingredient inventory. During the cooking process, the AI ​​agent monitors the cooking in real time and provides advice at the appropriate time. For example, it monitors the condition of the ingredients and the cooking procedure, and suggests alternative methods as needed. After the dish is completed, it provides a successful experience with the customized recipe and accumulates feedback for future use. Through this mechanism, users can improve their cooking skills and enjoy cooking with confidence. In addition, the AI ​​agent has a learning function and provides personalized support by offering advice tailored to the user's taste preferences. This reduces cooking failures and gives users the freedom to try new recipes. Furthermore, real-time feedback promotes user engagement, and continued use can be expected. For business owners, the introduction of new technology can expand the market and enable the provision of differentiated services. In this way, the cooking support system can improve users' cooking skills and maximize the enjoyment of cooking.

[0029] The cooking support system according to this embodiment comprises an analysis unit, a suggestion unit, a monitoring unit, and a feedback unit. The analysis unit analyzes the inventory and purchase history of ingredients. For example, the analysis unit automatically scans the inventory in the refrigerator and pantry and stores it in a database. The analysis unit can also analyze the user's purchase history and understand past purchasing patterns. For example, the analysis unit identifies ingredients that the user frequently purchases and updates their inventory status in real time. The suggestion unit proposes the optimal recipe based on the information analyzed by the analysis unit. For example, the suggestion unit proposes a nutritionally balanced recipe based on the user's taste preferences and ingredient inventory. The suggestion unit can also propose recipes according to the user's cooking skills. For example, the suggestion unit proposes simple recipes for beginners and complex recipes for advanced users. The monitoring unit monitors the cooking process in real time based on the recipe proposed by the suggestion unit and provides advice at the appropriate time. For example, the monitoring unit detects the state of ingredients during cooking using sensors and advises on appropriate cooking time and temperature. Furthermore, the monitoring unit can monitor the progress of the cooking procedure and notify the user of the next step. For example, the monitoring unit can detect when the ingredients are cooked and notify the user. The feedback unit provides a customized recipe based on the information monitored by the monitoring unit and accumulates feedback for future use. For example, the feedback unit collects evaluations of the dishes cooked by the user and reflects them in the next recipe suggestion. The feedback unit can also provide specific advice to support the improvement of the user's cooking skills. For example, the feedback unit points out areas where the user should improve and provides information that will be useful for future cooking. In this way, the cooking support system according to the embodiment can improve the user's cooking skills and maximize the enjoyment of cooking.

[0030] The analysis unit analyzes food inventory and purchase history. For example, it automatically scans the inventory in the refrigerator and pantry and stores it in a database. Specifically, it uses cameras and RFID sensors installed in the refrigerator and pantry to automatically recognize the type and quantity of food and record it in the database. This eliminates the need for users to manually check their inventory. The analysis unit can also analyze the user's purchase history and understand past purchasing patterns. For example, it can identify the food items that users frequently purchase and update their inventory status in real time. Furthermore, based on the user's purchase history, the analysis unit can predict food consumption trends and expiration dates and make suggestions to reduce waste. For example, it can reduce food waste by suggesting recipes that prioritize the use of food items nearing their expiration date. The analysis unit can also analyze the user's purchase frequency and consumption rate of food items and automatically generate a shopping list for the next trip. This allows users to shop efficiently and always have the necessary ingredients on hand. Through these functions, the analysis unit can support the user's food management and provide an efficient cooking environment.

[0031] The suggestion department proposes optimal recipes based on information analyzed by the analysis department. For example, the suggestion department proposes nutritionally balanced recipes based on the user's taste preferences and ingredient inventory. Specifically, it analyzes the user's past recipe selections and evaluations to learn their preferences. The suggestion department can also propose recipes tailored to the user's health condition and nutritional needs. For example, it can propose recipes using ingredients rich in specific nutrients or recipes that take calorie restrictions into consideration. Furthermore, the suggestion department can propose recipes tailored to the user's cooking skill level. For example, it can propose simple recipes for beginners and complex recipes for advanced cooks. This allows users to choose recipes that match their skill level and enjoy cooking. The suggestion department can also propose recipes tailored to the season and events. For example, by proposing recipes using seasonal ingredients or recipes tailored to special events, it can enrich the user's cooking experience. Through these functions, the suggestion department can provide users with optimal recipes and support both the enjoyment of cooking and their health.

[0032] The monitoring unit monitors the cooking process in real time based on the recipe proposed by the suggestion unit and provides advice at the appropriate time. For example, the monitoring unit detects the state of ingredients during cooking using sensors and advises on the appropriate cooking time and temperature. Specifically, it uses temperature and humidity sensors built into the cooking appliance to monitor the state of ingredients in real time. This allows the user to proceed with cooking at the appropriate time and prevent failures. The monitoring unit can also monitor the progress of the cooking procedure and notify the user of the next step. For example, the monitoring unit detects when the ingredients are cooked and notifies the user. Furthermore, the monitoring unit can detect abnormalities during cooking and issue a warning to the user. For example, if the temperature rises abnormally during cooking or if the ingredients are about to burn, it will warn the user and prompt them to take appropriate action. In this way, the monitoring unit can support the user in cooking safely and efficiently. In addition, the monitoring unit can accumulate data on the cooking process and use it for analysis and improvement at a later date. This allows the user to receive feedback to improve their cooking skills. Through these functions, the monitoring unit can support the user's cooking experience and increase the success rate of their cooking.

[0033] The Feedback Department provides successful experiences through customized recipes based on information monitored by the Monitoring Department, and accumulates feedback for future use. For example, the Feedback Department collects evaluations of dishes cooked by users and incorporates them into future recipe suggestions. Specifically, it provides an interface for users to evaluate the taste, appearance, and difficulty of dishes after cooking. This allows user evaluations to be accumulated in a database and used for future recipe suggestions. The Feedback Department can also provide specific advice to support the improvement of users' cooking skills. For example, the Feedback Department points out areas where users need improvement and provides information useful for future cooking. Furthermore, the Feedback Department can suggest more personalized recipes based on users' preferences and past evaluations. This makes it easier for users to find recipes that suit their tastes and to enjoy cooking. The Feedback Department can also analyze users' cooking history and provide plans for long-term skill improvement. For example, based on recipes that users have attempted in the past and their results, the Feedback Department provides advice on recipes to try next and ways to improve skills. This allows users to improve their cooking skills step by step. The feedback function, through these features, can improve users' cooking skills and maximize the enjoyment of cooking.

[0034] The analysis unit can analyze food inventory and purchase history. For example, the analysis unit can automatically scan the inventory in the refrigerator and save it to a database. The analysis unit can also analyze the user's purchase history and understand past purchasing patterns. For example, the analysis unit can identify the food items that the user frequently purchases and update their inventory status in real time. This allows the analysis unit to suggest optimal recipes based on the user's taste preferences and food inventory. Some or all of the above processes in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input the inventory data from the refrigerator into a generating AI and have the generating AI perform the inventory analysis.

[0035] The suggestion unit can provide advice tailored to the user's taste preferences. For example, if the user likes sweet things, the suggestion unit will suggest sweet recipes. Similarly, if the user dislikes spicy food, the suggestion unit can suggest recipes with reduced spiciness. This allows the suggestion unit to provide personalized support tailored to the user's taste preferences. Some or all of the above-described processes in the suggestion unit may be performed using AI, or not. For example, the suggestion unit can input the user's taste preference data into a generating AI and have the generating AI generate recipe suggestions.

[0036] The suggestion unit can propose alternative methods as needed. For example, if a particular ingredient is in short supply, the suggestion unit can suggest an alternative ingredient. It can also suggest a simpler alternative method if a particular cooking method is difficult. This allows the suggestion unit to increase cooking flexibility. Some or all of the above-described processes in the suggestion unit may be performed using AI, for example, or without AI. For example, the suggestion unit can input ingredient inventory data into a generating AI and have the generating AI propose alternative methods.

[0037] The suggestion unit can facilitate user engagement through real-time feedback. For example, the suggestion unit can provide real-time advice to the user during cooking and support the progress of the cooking process. The suggestion unit can also collect feedback after the user has completed cooking and incorporate it into the next recipe suggestion. This allows the suggestion unit to encourage continued use by the user. Some or all of the above processes in the suggestion unit may be performed using AI, for example, or not using AI. For example, the suggestion unit can input user feedback data into a generating AI and have the generating AI perform the task of facilitating engagement.

[0038] The analysis unit can analyze the user's past purchase history and select the optimal analysis method. For example, the analysis unit can prioritize analyzing the ingredients that the user frequently purchases and understand their inventory status. The analysis unit can also predict the ingredients that the user will purchase in a particular season based on their purchase history and incorporate this into the analysis. Furthermore, the analysis unit can analyze the user's past purchase patterns and predict and analyze the inventory of necessary ingredients. In this way, the analysis unit can select the optimal analysis method by analyzing the user's past purchase history. Some or all of the above processes in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input the user's purchase history data into a generating AI and have the generating AI select the optimal analysis method.

[0039] The analysis unit can filter food inventory based on the user's current health status and dietary restrictions. For example, if the user is on a diet, the analysis unit will prioritize analyzing low-calorie foods. Furthermore, if the user has allergies, the analysis unit can exclude foods containing allergens from the analysis. Additionally, if the user requires a specific nutrient, the analysis unit can prioritize analyzing foods rich in that nutrient. This allows the analysis unit to perform a more appropriate food inventory analysis by filtering based on the user's health status and dietary restrictions. Some or all of the above processing in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input the user's health status data into a generating AI and have the generating AI perform the filtering.

[0040] The analysis unit can prioritize the analysis of highly relevant ingredients when analyzing ingredient inventory, taking into account the user's geographical location. For example, the analysis unit can prioritize the analysis of ingredients commonly used in the user's area. It can also prioritize the analysis of ingredients available at nearby supermarkets based on the user's location. Furthermore, based on the user's location, the analysis unit can prioritize the analysis of local specialty products. This allows the analysis unit to analyze ingredients more appropriately by prioritizing highly relevant ingredients while considering the user's geographical location. Some or all of the above processing in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input the user's geographical location into a generating AI and have the generating AI perform the analysis of highly relevant ingredients.

[0041] The analysis unit can analyze users' social media activity and identify relevant ingredients when analyzing ingredient inventory. For example, the analysis unit can prioritize analyzing ingredients used in recipes shared by users on social media. It can also prioritize analyzing ingredients featured on cooking accounts that users follow. Furthermore, it can prioritize analyzing ingredients used in cooking posts that users have "liked." In this way, the analysis unit can identify relevant ingredients by analyzing users' social media activity. Some or all of the above processing in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input user social media data into a generating AI and have the generating AI perform the analysis of relevant ingredients.

[0042] The suggestion unit can adjust the level of detail in its recipe suggestions based on the importance of the ingredients. For example, it can provide detailed cooking instructions for key ingredients and simplify instructions for auxiliary ingredients. Alternatively, it can provide detailed instructions on how to cook key ingredients and only an overview for other ingredients. Furthermore, it can also provide detailed instructions on how to select and store key ingredients. This allows the suggestion unit to adjust the level of detail in its suggestions based on the importance of the ingredients. Some or all of the above processing in the suggestion unit may be performed using AI, for example, or not. For example, the suggestion unit can input ingredient importance data into a generating AI and have the generating AI perform the adjustment of the level of detail in its suggestions.

[0043] The suggestion unit can apply different suggestion algorithms depending on the category of ingredients when suggesting recipes. For example, in the case of vegetable dishes, the suggestion unit can suggest based on nutritional value and cooking method. In the case of meat dishes, the suggestion unit can also suggest based on the cut of meat and cooking time. Furthermore, in the case of desserts, the suggestion unit can also suggest based on sweetness and calories. In this way, the suggestion unit can provide more appropriate recipe suggestions by applying different suggestion algorithms depending on the category of ingredients. Some or all of the above processing in the suggestion unit may be performed using AI, for example, or without AI. For example, the suggestion unit can input ingredient category data into a generating AI and have the generating AI execute the application of the suggestion algorithm.

[0044] The suggestion unit can determine the priority of recipe suggestions based on the freshness of the ingredients. For example, the suggestion unit can suggest recipes that prioritize the use of fresh ingredients. It can also suggest recipes that help use up ingredients that are starting to lose their freshness quickly. Furthermore, it can suggest recipes that use ingredients that can be frozen. By determining the priority of suggestions based on the freshness of the ingredients, the suggestion unit can provide more appropriate recipe suggestions. Some or all of the above processing in the suggestion unit may be performed using AI, for example, or without AI. For example, the suggestion unit can input ingredient freshness data into a generating AI and have the generating AI perform the task of determining the priority of suggestions.

[0045] The suggestion unit can adjust the order of suggestions based on the relationships between ingredients when suggesting recipes. For example, the suggestion unit can suggest side dishes related to the main dish. It can also suggest multiple recipes using the same ingredients. Furthermore, the suggestion unit can make suggestions considering the synergistic effects of ingredient combinations. This allows the suggestion unit to make more appropriate recipe suggestions by adjusting the order of suggestions based on the relationships between ingredients. Some or all of the above processing in the suggestion unit may be performed using AI, for example, or without AI. For example, the suggestion unit can input ingredient relationship data into a generating AI and have the generating AI perform the adjustment of the suggestion order.

[0046] The monitoring unit can improve the accuracy of its monitoring by considering the interrelationships between cooking processes. For example, if multiple cooking processes are proceeding simultaneously, the monitoring unit can monitor the progress of each process in real time. The monitoring unit can also consider the order of cooking processes and provide appropriate advice on when to proceed to the next process. Furthermore, the monitoring unit can monitor the state of ingredients during cooking and suggest appropriate cooking times. In this way, the monitoring unit can improve the accuracy of its monitoring by considering the interrelationships between cooking processes, enabling more accurate monitoring. Some or all of the above-described processes in the monitoring unit may be performed using AI, for example, or without AI. For example, the monitoring unit can input cooking process data into a generating AI and have the generating AI perform the task of improving monitoring accuracy.

[0047] The monitoring unit can perform monitoring while considering the cook's attribute information. For example, the monitoring unit can adjust the frequency and content of advice according to the cook's experience level. The monitoring unit can also suggest appropriate cooking methods considering the cook's age and health condition. Furthermore, the monitoring unit can provide personalized advice based on the cook's preferences and past cooking history. As a result, the monitoring unit can perform more appropriate monitoring by considering the cook's attribute information. Some or all of the above processing in the monitoring unit may be performed using AI, for example, or without AI. For example, the monitoring unit can input the cook's attribute information into a generating AI and have the generating AI perform the adjustments to the monitoring.

[0048] The monitoring unit can perform monitoring while considering the geographical distribution of cooking. For example, the monitoring unit can perform monitoring while considering regional cooking methods and the characteristics of ingredients. The monitoring unit can also suggest cooking methods that are appropriate to the regional climate and season. Furthermore, the monitoring unit can monitor cooking methods based on regional culture and traditions. As a result, the monitoring unit can perform more appropriate monitoring by considering the geographical distribution of cooking. Some or all of the above processing in the monitoring unit may be performed using AI, for example, or without AI. For example, the monitoring unit can input regional cooking data into a generating AI and have the generating AI perform adjustments to the monitoring.

[0049] The monitoring unit can improve the accuracy of its monitoring by referring to relevant cooking literature during monitoring. For example, the monitoring unit performs monitoring by referring to the latest research results on the ingredients being cooked. The monitoring unit can also suggest the optimal cooking method based on past literature on cooking methods. Furthermore, the monitoring unit can suggest appropriate cooking time and temperature by referring to literature on the characteristics of the ingredients being cooked. In this way, the monitoring unit can improve the accuracy of its monitoring by referring to relevant cooking literature, enabling more appropriate monitoring. Some or all of the above processing in the monitoring unit may be performed using AI, for example, or without AI. For example, the monitoring unit can input cooking-related literature data into a generating AI and have the generating AI perform the improvement of monitoring accuracy.

[0050] The feedback unit can predict current feedback by referring to past feedback data during the feedback process. For example, the feedback unit can predict feedback on the current cooking based on feedback the user has received in the past. The feedback unit can also analyze the user's growth and areas for improvement from past feedback data and reflect this in the current feedback. Furthermore, the feedback unit can refer to past feedback data to highlight points that the user should pay particular attention to. This allows the feedback unit to provide more appropriate feedback by referring to past feedback data to predict current feedback. Some or all of the above processing in the feedback unit may be performed using AI, for example, or without AI. For example, the feedback unit can input past feedback data into a generating AI and have the generating AI perform the current feedback prediction.

[0051] The feedback unit can apply different feedback analysis methods to each cooking category when providing feedback. For example, in the case of vegetable dishes, the feedback unit provides feedback based on nutritional value and cooking method. In the case of meat dishes, the feedback unit can also provide feedback based on doneness and seasoning. Furthermore, in the case of desserts, the feedback unit can provide feedback based on sweetness and texture. This allows the feedback unit to provide more appropriate feedback by applying different feedback analysis methods to each cooking category. Some or all of the above processing in the feedback unit may be performed using AI, for example, or without AI. For example, the feedback unit can input cooking category data into a generating AI and have the generating AI perform the application of the feedback analysis method.

[0052] The feedback unit can analyze changes in feedback based on the submission timing of the recipe. For example, the feedback unit can adjust the content and importance of the feedback according to the submission timing of the recipe. The feedback unit can also compare past submission times with current submission times and analyze changes in feedback. Furthermore, the feedback unit can reflect user growth and areas for improvement in the feedback based on the submission timing. This allows the feedback unit to provide more appropriate feedback by analyzing changes in feedback based on the submission timing of the recipe. Some or all of the above processing in the feedback unit may be performed using AI, for example, or without AI. For example, the feedback unit can input submission timing data into a generating AI and have the generating AI perform the analysis of changes in feedback.

[0053] The feedback unit can analyze feedback by referring to relevant market data related to cooking. For example, the feedback unit can reflect current cooking methods and ingredient trends in the feedback based on market data. The feedback unit can also suggest new cooking methods and ingredients that the user should try based on market data. Furthermore, the feedback unit can refer to market data and analyze whether the user's cooking method is in line with market trends. This allows the feedback unit to provide more appropriate feedback by referring to relevant market data related to cooking and analyzing the feedback. Some or all of the above processing in the feedback unit may be performed using AI, for example, or not using AI. For example, the feedback unit can input market data into a generating AI and have the generating AI perform the feedback analysis.

[0054] The advice unit can adjust the level of detail in its advice based on the importance of the cooking process. For example, it can provide detailed advice on key cooking steps and simplify auxiliary steps. It can also explain the procedures for important cooking steps in detail and provide only an overview for other steps. Furthermore, it can explain the key points and precautions for key cooking steps in detail. This allows the advice unit to provide more appropriate advice by adjusting the level of detail based on the importance of the cooking process. Some or all of the above processing in the advice unit may be performed using AI, for example, or not. For example, the advice unit can input cooking importance data into a generating AI and have the generating AI perform the adjustment of the level of detail in the advice.

[0055] The advice unit can apply different advice algorithms depending on the cooking category when providing advice. For example, in the case of vegetable dishes, the advice unit provides advice based on nutritional value and cooking method. In the case of meat dishes, the advice unit can also provide advice based on the cut of meat and cooking time. Furthermore, in the case of desserts, the advice unit can provide advice based on sweetness and calories. This allows the advice unit to provide more appropriate advice by applying different advice algorithms depending on the cooking category. Some or all of the above processing in the advice unit may be performed using AI, for example, or without AI. For example, the advice unit can input cooking category data into a generating AI and have the generating AI execute the application of the advice algorithm.

[0056] The advice unit can prioritize advice based on the submission date of the recipe. For example, if the submission date is approaching, the advice unit will prioritize providing important advice. If the submission date is far off, the advice unit can also provide detailed advice. Furthermore, the advice unit can adjust the content and importance of the advice based on the submission date. This allows the advice unit to provide more appropriate advice by prioritizing advice based on the submission date of the recipe. Some or all of the above processing in the advice unit may be performed using AI, for example, or not using AI. For example, the advice unit can input submission date data into a generating AI and have the generating AI perform the priority determination of advice.

[0057] The advice unit can adjust the order of advice based on the relevance of the cooking process. For example, it may prioritize advice on side dishes related to the main dish. It can also sequentially provide advice on multiple cooking steps using the same ingredients. Furthermore, it can adjust the order of advice considering the interrelationships of the cooking steps. This allows the advice unit to provide more appropriate advice by adjusting the order of advice based on the relevance of the cooking process. Some or all of the above processing in the advice unit may be performed using AI, for example, or without AI. For example, the advice unit can input cooking relevance data into a generating AI and have the generating AI perform the adjustment of the order of advice.

[0058] The alternative suggestion unit can adjust the level of detail of alternative methods based on the importance of the cooking process when suggesting alternative methods. For example, the alternative suggestion unit can provide detailed alternative methods for major cooking processes and simplify auxiliary processes. Alternatively, the alternative suggestion unit can provide detailed explanations of alternative methods for important cooking processes and only outlines other processes. Furthermore, the alternative suggestion unit can also provide detailed explanations of key points and considerations for alternative methods of major cooking processes. This allows the alternative suggestion unit to suggest more appropriate alternative methods by adjusting the level of detail of alternative methods based on the importance of the cooking process. Some or all of the above processing in the alternative suggestion unit may be performed using AI, for example, or not. For example, the alternative suggestion unit can input cooking importance data into a generating AI and have the generating AI perform the adjustment of the level of detail of the alternative methods.

[0059] The alternative suggestion unit can apply different alternative method algorithms depending on the cooking category when suggesting alternative methods. For example, in the case of vegetable dishes, the alternative suggestion unit can suggest alternative methods based on nutritional value and cooking method. In the case of meat dishes, the alternative suggestion unit can also suggest alternative methods based on the cut of meat and cooking time. Furthermore, in the case of desserts, the alternative suggestion unit can also suggest alternative methods based on sweetness and calories. In this way, the alternative suggestion unit can suggest more appropriate alternative methods by applying different alternative method algorithms depending on the cooking category. Some or all of the above processing in the alternative suggestion unit may be performed using AI, for example, or without AI. For example, the alternative suggestion unit can input cooking category data into a generating AI and have the generating AI execute the application of alternative method algorithms.

[0060] The alternative proposal unit can prioritize alternative methods based on the submission timing of the recipe when proposing alternative methods. For example, if the submission deadline is approaching, the alternative proposal unit will prioritize providing important alternative methods. If the submission deadline is far off, the alternative proposal unit can also provide detailed alternative methods. Furthermore, the alternative proposal unit can adjust the content and importance of alternative methods based on the submission timing. This allows the alternative proposal unit to propose more appropriate alternative methods by prioritizing them based on the submission timing of the recipe. Some or all of the above processing in the alternative proposal unit may be performed using AI, for example, or not using AI. For example, the alternative proposal unit can input submission timing data into a generating AI and have the generating AI perform the determination of alternative method priorities.

[0061] The alternative suggestion unit can adjust the order of alternative methods based on the relevance of the cooking process when suggesting alternative methods. For example, the alternative suggestion unit may prioritize providing alternative methods for side dishes related to the main dish. The alternative suggestion unit can also sequentially provide alternative methods for multiple cooking processes using the same ingredients. Furthermore, the alternative suggestion unit can adjust the order of alternative methods considering the interrelationships of the cooking processes. This allows the alternative suggestion unit to suggest more appropriate alternative methods by adjusting the order of alternative methods based on the relevance of the cooking process. Some or all of the above processing in the alternative suggestion unit may be performed using AI, for example, or without AI. For example, the alternative suggestion unit can input cooking relevance data into a generating AI and have the generating AI perform the order adjustment of alternative methods.

[0062] The engagement unit can select the optimal display method by referring to the user's past operation history when displaying engagement. For example, the engagement unit can prioritize providing display methods that the user has preferred to use in the past. The engagement unit can also select the most effective display method from the user's past operation history. Furthermore, the engagement unit can provide personalized display methods based on the user's operation history. As a result, the engagement unit can enable more appropriate engagement by selecting the optimal display method by referring to the user's past operation history. Some or all of the above processing in the engagement unit may be performed using AI, for example, or without AI. For example, the engagement unit can input past operation history data into a generating AI and have the generating AI select the optimal display method.

[0063] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0064] The analysis unit can analyze users' food consumption patterns and make suggestions considering the expiration dates of ingredients. For example, the analysis unit can prioritize analyzing the expiration dates of ingredients that users frequently consume and suggest recipes that use ingredients that are nearing their expiration date. The analysis unit can also suggest recipes that prioritize the use of ingredients with shorter expiration dates, while delaying the use of ingredients with longer expiration dates. Furthermore, the analysis unit can analyze users' consumption patterns and make suggestions to reduce food waste. As a result, the analysis unit can make more appropriate recipe suggestions by considering users' food consumption patterns.

[0065] The suggestion function can propose recipes while taking into account the user's food allergy information. For example, if a user is allergic to a specific ingredient, the suggestion function will propose a recipe that does not include that ingredient. The suggestion function can also propose recipes that substitute ingredients that may cause allergies. Furthermore, based on the user's allergy information, the suggestion function can also propose cooking methods that avoid allergies. This allows the suggestion function to propose safer and more appropriate recipes while taking the user's allergy information into consideration.

[0066] The monitoring unit can perform monitoring while considering the user's cooking environment. For example, the monitoring unit can suggest appropriate cooking methods considering the type of equipment and cooking utensils in the user's kitchen. Furthermore, the monitoring unit can adjust cooking time and temperature according to the user's cooking environment. In addition, the monitoring unit can monitor the progress of cooking based on the user's cooking environment and provide advice at the appropriate time. This allows the monitoring unit to perform more appropriate monitoring while considering the user's cooking environment.

[0067] The feedback unit can analyze the user's cooking history and suggest areas for improvement. For example, it can point out areas for improvement based on the user's evaluation of dishes they have cooked in the past. It can also provide advice on specific cooking methods or ingredient usage based on the user's cooking history. Furthermore, it can analyze the user's cooking history and provide information useful for future cooking. This allows the feedback unit to provide more appropriate feedback by taking the user's cooking history into consideration.

[0068] The analysis unit can perform inventory analysis while considering the user's food storage methods. For example, the analysis unit can differentiate between food items that require refrigeration and those that can be stored at room temperature. Furthermore, the analysis unit can prioritize the analysis of food items that can be frozen and suggest recipes that allow for long-term storage. In addition, the analysis unit can offer suggestions to prevent food spoilage based on the user's storage methods. This allows the analysis unit to perform more appropriate inventory analysis by considering the user's food storage methods.

[0069] The following briefly describes the processing flow for example form 1.

[0070] Step 1: The analysis unit analyzes food inventory and purchase history. The analysis unit automatically scans the inventory in the refrigerator and pantry and saves it to the database. It can also analyze the user's purchase history to understand past purchasing patterns. For example, it can identify the food items that the user frequently purchases and update their inventory status in real time. Step 2: The suggestion unit proposes the optimal recipe based on the information analyzed by the analysis unit. The suggestion unit proposes a nutritionally balanced recipe based on the user's taste preferences and ingredient inventory. It can also propose recipes tailored to the user's cooking skill level. For example, it can suggest simple recipes for beginners and complex recipes for advanced cooks. Step 3: The monitoring unit monitors the cooking process in real time based on the recipe proposed by the suggestion unit and provides advice at the appropriate time. The monitoring unit detects the state of the ingredients during cooking using sensors and advises on the appropriate cooking time and temperature. It can also monitor the progress of the cooking procedure and notify the user of the next step. For example, it can detect when the ingredients are cooked and notify the user. Step 4: The Feedback Department provides a customized recipe based on the information monitored by the Monitoring Department, accumulating feedback for future reference. The Feedback Department collects evaluations of the dishes cooked by the user and incorporates them into future recipe suggestions. It can also provide specific advice to support the user in improving their cooking skills. For example, it can point out areas where the user needs improvement and provide information that will be useful for future cooking.

[0071] (Example of form 2) The cooking support system according to an embodiment of the present invention is a system that uses an AI agent to maximize the enjoyment of cooking and improve cooking skills. When a user starts cooking, the AI ​​agent automatically analyzes the user's ingredient inventory and purchase history. Next, the AI ​​agent suggests an optimal recipe based on the user's taste preferences and ingredient inventory. During the cooking process, the AI ​​agent monitors the cooking in real time and provides advice at the appropriate time. For example, it monitors the condition of the ingredients and the cooking procedure, and suggests alternative methods as needed. After the dish is completed, it provides a successful experience with the customized recipe and accumulates feedback for future use. Through this mechanism, users can improve their cooking skills and enjoy cooking with confidence. In addition, the AI ​​agent has a learning function and provides personalized support by offering advice tailored to the user's taste preferences. This reduces cooking failures and gives users the freedom to try new recipes. Furthermore, real-time feedback promotes user engagement, and continued use can be expected. For business owners, the introduction of new technology can expand the market and enable the provision of differentiated services. In this way, the cooking support system can improve users' cooking skills and maximize the enjoyment of cooking.

[0072] The cooking support system according to this embodiment comprises an analysis unit, a suggestion unit, a monitoring unit, and a feedback unit. The analysis unit analyzes the inventory and purchase history of ingredients. For example, the analysis unit automatically scans the inventory in the refrigerator and pantry and stores it in a database. The analysis unit can also analyze the user's purchase history and understand past purchasing patterns. For example, the analysis unit identifies ingredients that the user frequently purchases and updates their inventory status in real time. The suggestion unit proposes the optimal recipe based on the information analyzed by the analysis unit. For example, the suggestion unit proposes a nutritionally balanced recipe based on the user's taste preferences and ingredient inventory. The suggestion unit can also propose recipes according to the user's cooking skills. For example, the suggestion unit proposes simple recipes for beginners and complex recipes for advanced users. The monitoring unit monitors the cooking process in real time based on the recipe proposed by the suggestion unit and provides advice at the appropriate time. For example, the monitoring unit detects the state of ingredients during cooking using sensors and advises on appropriate cooking time and temperature. Furthermore, the monitoring unit can monitor the progress of the cooking procedure and notify the user of the next step. For example, the monitoring unit can detect when the ingredients are cooked and notify the user. The feedback unit provides a customized recipe based on the information monitored by the monitoring unit and accumulates feedback for future use. For example, the feedback unit collects evaluations of the dishes cooked by the user and reflects them in the next recipe suggestion. The feedback unit can also provide specific advice to support the improvement of the user's cooking skills. For example, the feedback unit points out areas where the user should improve and provides information that will be useful for future cooking. In this way, the cooking support system according to the embodiment can improve the user's cooking skills and maximize the enjoyment of cooking.

[0073] The analysis unit analyzes food inventory and purchase history. For example, it automatically scans the inventory in the refrigerator and pantry and stores it in a database. Specifically, it uses cameras and RFID sensors installed in the refrigerator and pantry to automatically recognize the type and quantity of food and record it in the database. This eliminates the need for users to manually check their inventory. The analysis unit can also analyze the user's purchase history and understand past purchasing patterns. For example, it can identify the food items that users frequently purchase and update their inventory status in real time. Furthermore, based on the user's purchase history, the analysis unit can predict food consumption trends and expiration dates and make suggestions to reduce waste. For example, it can reduce food waste by suggesting recipes that prioritize the use of food items nearing their expiration date. The analysis unit can also analyze the user's purchase frequency and consumption rate of food items and automatically generate a shopping list for the next trip. This allows users to shop efficiently and always have the necessary ingredients on hand. Through these functions, the analysis unit can support the user's food management and provide an efficient cooking environment.

[0074] The suggestion department proposes optimal recipes based on information analyzed by the analysis department. For example, the suggestion department proposes nutritionally balanced recipes based on the user's taste preferences and ingredient inventory. Specifically, it analyzes the user's past recipe selections and evaluations to learn their preferences. The suggestion department can also propose recipes tailored to the user's health condition and nutritional needs. For example, it can propose recipes using ingredients rich in specific nutrients or recipes that take calorie restrictions into consideration. Furthermore, the suggestion department can propose recipes tailored to the user's cooking skill level. For example, it can propose simple recipes for beginners and complex recipes for advanced cooks. This allows users to choose recipes that match their skill level and enjoy cooking. The suggestion department can also propose recipes tailored to the season and events. For example, by proposing recipes using seasonal ingredients or recipes tailored to special events, it can enrich the user's cooking experience. Through these functions, the suggestion department can provide users with optimal recipes and support both the enjoyment of cooking and their health.

[0075] The monitoring unit monitors the cooking process in real time based on the recipe proposed by the suggestion unit and provides advice at the appropriate time. For example, the monitoring unit detects the state of ingredients during cooking using sensors and advises on the appropriate cooking time and temperature. Specifically, it uses temperature and humidity sensors built into the cooking appliance to monitor the state of ingredients in real time. This allows the user to proceed with cooking at the appropriate time and prevent failures. The monitoring unit can also monitor the progress of the cooking procedure and notify the user of the next step. For example, the monitoring unit detects when the ingredients are cooked and notifies the user. Furthermore, the monitoring unit can detect abnormalities during cooking and issue a warning to the user. For example, if the temperature rises abnormally during cooking or if the ingredients are about to burn, it will warn the user and prompt them to take appropriate action. In this way, the monitoring unit can support the user in cooking safely and efficiently. In addition, the monitoring unit can accumulate data on the cooking process and use it for analysis and improvement at a later date. This allows the user to receive feedback to improve their cooking skills. Through these functions, the monitoring unit can support the user's cooking experience and increase the success rate of their cooking.

[0076] The Feedback Department provides successful experiences through customized recipes based on information monitored by the Monitoring Department, and accumulates feedback for future use. For example, the Feedback Department collects evaluations of dishes cooked by users and incorporates them into future recipe suggestions. Specifically, it provides an interface for users to evaluate the taste, appearance, and difficulty of dishes after cooking. This allows user evaluations to be accumulated in a database and used for future recipe suggestions. The Feedback Department can also provide specific advice to support the improvement of users' cooking skills. For example, the Feedback Department points out areas where users need improvement and provides information useful for future cooking. Furthermore, the Feedback Department can suggest more personalized recipes based on users' preferences and past evaluations. This makes it easier for users to find recipes that suit their tastes and to enjoy cooking. The Feedback Department can also analyze users' cooking history and provide plans for long-term skill improvement. For example, based on recipes that users have attempted in the past and their results, the Feedback Department provides advice on recipes to try next and ways to improve skills. This allows users to improve their cooking skills step by step. The feedback function, through these features, can improve users' cooking skills and maximize the enjoyment of cooking.

[0077] The analysis unit can analyze food inventory and purchase history. For example, the analysis unit can automatically scan the inventory in the refrigerator and save it to a database. The analysis unit can also analyze the user's purchase history and understand past purchasing patterns. For example, the analysis unit can identify the food items that the user frequently purchases and update their inventory status in real time. This allows the analysis unit to suggest optimal recipes based on the user's taste preferences and food inventory. Some or all of the above processes in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input the inventory data from the refrigerator into a generating AI and have the generating AI perform the inventory analysis.

[0078] The suggestion unit can provide advice tailored to the user's taste preferences. For example, if the user likes sweet things, the suggestion unit will suggest sweet recipes. Similarly, if the user dislikes spicy food, the suggestion unit can suggest recipes with reduced spiciness. This allows the suggestion unit to provide personalized support tailored to the user's taste preferences. Some or all of the above-described processes in the suggestion unit may be performed using AI, or not. For example, the suggestion unit can input the user's taste preference data into a generating AI and have the generating AI generate recipe suggestions.

[0079] The suggestion unit can propose alternative methods as needed. For example, if a particular ingredient is in short supply, the suggestion unit can suggest an alternative ingredient. It can also suggest a simpler alternative method if a particular cooking method is difficult. This allows the suggestion unit to increase cooking flexibility. Some or all of the above-described processes in the suggestion unit may be performed using AI, for example, or without AI. For example, the suggestion unit can input ingredient inventory data into a generating AI and have the generating AI propose alternative methods.

[0080] The suggestion unit can facilitate user engagement through real-time feedback. For example, the suggestion unit can provide real-time advice to the user during cooking and support the progress of the cooking process. The suggestion unit can also collect feedback after the user has completed cooking and incorporate it into the next recipe suggestion. This allows the suggestion unit to encourage continued use by the user. Some or all of the above processes in the suggestion unit may be performed using AI, for example, or not using AI. For example, the suggestion unit can input user feedback data into a generating AI and have the generating AI perform the task of facilitating engagement.

[0081] The analysis unit can estimate the user's emotions and adjust the timing of ingredient inventory analysis based on the estimated emotions. For example, if the user is stressed, the analysis unit can quickly analyze ingredient inventory and immediately begin making suggestions. If the user is relaxed, the analysis unit can also perform a detailed inventory analysis and provide more options. Furthermore, if the user is in a hurry, the analysis unit can prioritize analyzing only the most important ingredients and quickly make suggestions. In this way, the analysis unit can adjust the timing of ingredient inventory analysis according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the analysis unit may be performed using AI, for example, or not using AI. For example, the analysis unit can input user emotion data into a generative AI and have the generative AI perform the adjustment of inventory analysis timing.

[0082] The analysis unit can analyze the user's past purchase history and select the optimal analysis method. For example, the analysis unit can prioritize analyzing the ingredients that the user frequently purchases and understand their inventory status. The analysis unit can also predict the ingredients that the user will purchase in a particular season based on their purchase history and incorporate this into the analysis. Furthermore, the analysis unit can analyze the user's past purchase patterns and predict and analyze the inventory of necessary ingredients. In this way, the analysis unit can select the optimal analysis method by analyzing the user's past purchase history. Some or all of the above processes in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input the user's purchase history data into a generating AI and have the generating AI select the optimal analysis method.

[0083] The analysis unit can filter food inventory based on the user's current health status and dietary restrictions. For example, if the user is on a diet, the analysis unit will prioritize analyzing low-calorie foods. Furthermore, if the user has allergies, the analysis unit can exclude foods containing allergens from the analysis. Additionally, if the user requires a specific nutrient, the analysis unit can prioritize analyzing foods rich in that nutrient. This allows the analysis unit to perform a more appropriate food inventory analysis by filtering based on the user's health status and dietary restrictions. Some or all of the above processing in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input the user's health status data into a generating AI and have the generating AI perform the filtering.

[0084] The analysis unit can estimate the user's emotions and determine the priority of ingredients to analyze based on the estimated emotions. For example, if the user is stressed, the analysis unit may prioritize ingredients with relaxing effects. It can also prioritize ingredients suitable for energy replenishment if the user is tired. Furthermore, if the user is enjoying themselves, the analysis unit may prioritize new or unusual ingredients. This allows the analysis unit to determine the priority of ingredients to analyze according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processes in the analysis unit may be performed using AI, or not. For example, the analysis unit can input user emotion data into a generative AI and have the generative AI determine the priority of ingredients.

[0085] The analysis unit can prioritize the analysis of highly relevant ingredients when analyzing ingredient inventory, taking into account the user's geographical location. For example, the analysis unit can prioritize the analysis of ingredients commonly used in the user's area. It can also prioritize the analysis of ingredients available at nearby supermarkets based on the user's location. Furthermore, based on the user's location, the analysis unit can prioritize the analysis of local specialty products. This allows the analysis unit to analyze ingredients more appropriately by prioritizing highly relevant ingredients while considering the user's geographical location. Some or all of the above processing in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input the user's geographical location into a generating AI and have the generating AI perform the analysis of highly relevant ingredients.

[0086] The analysis unit can analyze users' social media activity and identify relevant ingredients when analyzing ingredient inventory. For example, the analysis unit can prioritize analyzing ingredients used in recipes shared by users on social media. It can also prioritize analyzing ingredients featured on cooking accounts that users follow. Furthermore, it can prioritize analyzing ingredients used in cooking posts that users have "liked." In this way, the analysis unit can identify relevant ingredients by analyzing users' social media activity. Some or all of the above processing in the analysis unit may be performed using AI, for example, or without AI. For example, the analysis unit can input user social media data into a generating AI and have the generating AI perform the analysis of relevant ingredients.

[0087] The suggestion unit can estimate the user's emotions and adjust its recipe suggestion method based on the estimated emotions. For example, if the user is feeling stressed, the suggestion unit can suggest a simple and easy recipe. If the user is relaxed, it can suggest a recipe that can be enjoyed at a leisurely pace. Furthermore, if the user is in a hurry, it can suggest a recipe that can be made in a short amount of time. In this way, the suggestion unit can adjust its recipe suggestion method according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the suggestion unit may be performed using AI, for example, or not using AI. For example, the suggestion unit can input user emotion data into a generative AI and have the generative AI adjust the recipe suggestion method.

[0088] The suggestion unit can adjust the level of detail in its recipe suggestions based on the importance of the ingredients. For example, it can provide detailed cooking instructions for key ingredients and simplify instructions for auxiliary ingredients. Alternatively, it can provide detailed instructions on how to cook key ingredients and only an overview for other ingredients. Furthermore, it can also provide detailed instructions on how to select and store key ingredients. This allows the suggestion unit to adjust the level of detail in its suggestions based on the importance of the ingredients. Some or all of the above processing in the suggestion unit may be performed using AI, for example, or not. For example, the suggestion unit can input ingredient importance data into a generating AI and have the generating AI perform the adjustment of the level of detail in its suggestions.

[0089] The suggestion unit can apply different suggestion algorithms depending on the category of ingredients when suggesting recipes. For example, in the case of vegetable dishes, the suggestion unit can suggest based on nutritional value and cooking method. In the case of meat dishes, the suggestion unit can also suggest based on the cut of meat and cooking time. Furthermore, in the case of desserts, the suggestion unit can also suggest based on sweetness and calories. In this way, the suggestion unit can provide more appropriate recipe suggestions by applying different suggestion algorithms depending on the category of ingredients. Some or all of the above processing in the suggestion unit may be performed using AI, for example, or without AI. For example, the suggestion unit can input ingredient category data into a generating AI and have the generating AI execute the application of the suggestion algorithm.

[0090] The suggestion unit can estimate the user's emotions and adjust the length of its suggestions based on those emotions. For example, if the user is stressed, the suggestion unit can provide short, concise suggestions. If the user is relaxed, it can provide longer suggestions with more detailed explanations. Furthermore, if the user is in a hurry, it can provide quick and concise suggestions. This allows the suggestion unit to adjust the length of its suggestions according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the processing described above in the suggestion unit may be performed using AI or not. For example, the suggestion unit can input user emotion data into a generative AI and have the generative AI adjust the length of its suggestions.

[0091] The suggestion unit can determine the priority of recipe suggestions based on the freshness of the ingredients. For example, the suggestion unit can suggest recipes that prioritize the use of fresh ingredients. It can also suggest recipes that help use up ingredients that are starting to lose their freshness quickly. Furthermore, it can suggest recipes that use ingredients that can be frozen. By determining the priority of suggestions based on the freshness of the ingredients, the suggestion unit can provide more appropriate recipe suggestions. Some or all of the above processing in the suggestion unit may be performed using AI, for example, or without AI. For example, the suggestion unit can input ingredient freshness data into a generating AI and have the generating AI perform the task of determining the priority of suggestions.

[0092] The suggestion unit can adjust the order of suggestions based on the relationships between ingredients when suggesting recipes. For example, the suggestion unit can suggest side dishes related to the main dish. It can also suggest multiple recipes using the same ingredients. Furthermore, the suggestion unit can make suggestions considering the synergistic effects of ingredient combinations. This allows the suggestion unit to make more appropriate recipe suggestions by adjusting the order of suggestions based on the relationships between ingredients. Some or all of the above processing in the suggestion unit may be performed using AI, for example, or without AI. For example, the suggestion unit can input ingredient relationship data into a generating AI and have the generating AI perform the adjustment of the suggestion order.

[0093] The monitoring unit can estimate the user's emotions and adjust the monitoring criteria based on the estimated emotions. For example, if the user is stressed, the monitoring unit can closely monitor the progress of cooking and provide frequent advice. If the user is relaxed, the monitoring unit can broadly monitor the progress of cooking and provide advice only when necessary. Furthermore, if the user is in a hurry, the monitoring unit can monitor only the important points and provide quick advice. In this way, the monitoring unit can adjust the monitoring criteria according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the monitoring unit may be performed using AI or not using AI. For example, the monitoring unit can input user emotion data into the generative AI and have the generative AI adjust the monitoring criteria.

[0094] The monitoring unit can improve the accuracy of its monitoring by considering the interrelationships between cooking processes. For example, if multiple cooking processes are proceeding simultaneously, the monitoring unit can monitor the progress of each process in real time. The monitoring unit can also consider the order of cooking processes and provide appropriate advice on when to proceed to the next process. Furthermore, the monitoring unit can monitor the state of ingredients during cooking and suggest appropriate cooking times. In this way, the monitoring unit can improve the accuracy of its monitoring by considering the interrelationships between cooking processes, enabling more accurate monitoring. Some or all of the above-described processes in the monitoring unit may be performed using AI, for example, or without AI. For example, the monitoring unit can input cooking process data into a generating AI and have the generating AI perform the task of improving monitoring accuracy.

[0095] The monitoring unit can perform monitoring while considering the cook's attribute information. For example, the monitoring unit can adjust the frequency and content of advice according to the cook's experience level. The monitoring unit can also suggest appropriate cooking methods considering the cook's age and health condition. Furthermore, the monitoring unit can provide personalized advice based on the cook's preferences and past cooking history. As a result, the monitoring unit can perform more appropriate monitoring by considering the cook's attribute information. Some or all of the above processing in the monitoring unit may be performed using AI, for example, or without AI. For example, the monitoring unit can input the cook's attribute information into a generating AI and have the generating AI perform the adjustments to the monitoring.

[0096] The monitoring unit can estimate the user's emotions and adjust the order in which monitoring results are displayed based on the estimated emotions. For example, if the user is stressed, the monitoring unit can prioritize displaying important information. If the user is relaxed, the monitoring unit can also display detailed information in a sequential manner. Furthermore, if the user is in a hurry, the monitoring unit can quickly display concise information. This allows the monitoring unit to adjust the order in which monitoring results are displayed according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processing in the monitoring unit may be performed using AI, or not. For example, the monitoring unit can input user emotion data into a generative AI and have the generative AI adjust the display order of monitoring results.

[0097] The monitoring unit can perform monitoring while considering the geographical distribution of cooking. For example, the monitoring unit can perform monitoring while considering regional cooking methods and the characteristics of ingredients. The monitoring unit can also suggest cooking methods that are appropriate to the regional climate and season. Furthermore, the monitoring unit can monitor cooking methods based on regional culture and traditions. As a result, the monitoring unit can perform more appropriate monitoring by considering the geographical distribution of cooking. Some or all of the above processing in the monitoring unit may be performed using AI, for example, or without AI. For example, the monitoring unit can input regional cooking data into a generating AI and have the generating AI perform adjustments to the monitoring.

[0098] The monitoring unit can improve the accuracy of its monitoring by referring to relevant cooking literature during monitoring. For example, the monitoring unit performs monitoring by referring to the latest research results on the ingredients being cooked. The monitoring unit can also suggest the optimal cooking method based on past literature on cooking methods. Furthermore, the monitoring unit can suggest appropriate cooking time and temperature by referring to literature on the characteristics of the ingredients being cooked. In this way, the monitoring unit can improve the accuracy of its monitoring by referring to relevant cooking literature, enabling more appropriate monitoring. Some or all of the above processing in the monitoring unit may be performed using AI, for example, or without AI. For example, the monitoring unit can input cooking-related literature data into a generating AI and have the generating AI perform the improvement of monitoring accuracy.

[0099] The feedback unit can estimate the user's emotions and adjust how the feedback is displayed based on the estimated emotions. For example, if the user is stressed, the feedback unit can display concise and positive feedback. If the user is relaxed, the feedback unit can also display detailed feedback. Furthermore, if the user is in a hurry, the feedback unit can quickly display concise and to-the-point feedback. In this way, the feedback unit can adjust how the feedback is displayed according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. The generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the feedback unit may be performed using AI, for example, or not using AI. For example, the feedback unit can input user emotion data into the generative AI and have the generative AI adjust how the feedback is displayed.

[0100] The feedback unit can predict current feedback by referring to past feedback data during the feedback process. For example, the feedback unit can predict feedback on the current cooking based on feedback the user has received in the past. The feedback unit can also analyze the user's growth and areas for improvement from past feedback data and reflect this in the current feedback. Furthermore, the feedback unit can refer to past feedback data to highlight points that the user should pay particular attention to. This allows the feedback unit to provide more appropriate feedback by referring to past feedback data to predict current feedback. Some or all of the above processing in the feedback unit may be performed using AI, for example, or without AI. For example, the feedback unit can input past feedback data into a generating AI and have the generating AI perform the current feedback prediction.

[0101] The feedback unit can apply different feedback analysis methods to each cooking category when providing feedback. For example, in the case of vegetable dishes, the feedback unit provides feedback based on nutritional value and cooking method. In the case of meat dishes, the feedback unit can also provide feedback based on doneness and seasoning. Furthermore, in the case of desserts, the feedback unit can provide feedback based on sweetness and texture. This allows the feedback unit to provide more appropriate feedback by applying different feedback analysis methods to each cooking category. Some or all of the above processing in the feedback unit may be performed using AI, for example, or without AI. For example, the feedback unit can input cooking category data into a generating AI and have the generating AI perform the application of the feedback analysis method.

[0102] The feedback unit can estimate the user's emotions and adjust the importance of the feedback based on the estimated emotions. For example, if the user is stressed, the feedback unit can display only important feedback. It can also display detailed feedback if the user is relaxed. Furthermore, if the user is in a hurry, the feedback unit can quickly display concise feedback. This allows the feedback unit to adjust the importance of feedback according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the feedback unit may be performed using AI, or not. For example, the feedback unit can input user emotion data into a generative AI and have the generative AI adjust the importance of the feedback.

[0103] The feedback unit can analyze changes in feedback based on the submission timing of the recipe. For example, the feedback unit can adjust the content and importance of the feedback according to the submission timing of the recipe. The feedback unit can also compare past submission times with current submission times and analyze changes in feedback. Furthermore, the feedback unit can reflect user growth and areas for improvement in the feedback based on the submission timing. This allows the feedback unit to provide more appropriate feedback by analyzing changes in feedback based on the submission timing of the recipe. Some or all of the above processing in the feedback unit may be performed using AI, for example, or without AI. For example, the feedback unit can input submission timing data into a generating AI and have the generating AI perform the analysis of changes in feedback.

[0104] The feedback unit can analyze feedback by referring to relevant market data related to cooking. For example, the feedback unit can reflect current cooking methods and ingredient trends in the feedback based on market data. The feedback unit can also suggest new cooking methods and ingredients that the user should try based on market data. Furthermore, the feedback unit can refer to market data and analyze whether the user's cooking method is in line with market trends. This allows the feedback unit to provide more appropriate feedback by referring to relevant market data related to cooking and analyzing the feedback. Some or all of the above processing in the feedback unit may be performed using AI, for example, or not using AI. For example, the feedback unit can input market data into a generating AI and have the generating AI perform the feedback analysis.

[0105] The advice unit can estimate the user's emotions and adjust the way it expresses advice based on the estimated emotions. For example, if the user is stressed, the advice unit can provide concise and positive advice. If the user is relaxed, the advice unit can also provide detailed advice. Furthermore, if the user is in a hurry, the advice unit can provide concise and quick advice. In this way, the advice unit can adjust the way it expresses advice according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. The generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the advice unit may be performed using AI, or not using AI. For example, the advice unit can input user emotion data into the generative AI and have the generative AI adjust the way it expresses the advice.

[0106] The advice unit can adjust the level of detail in its advice based on the importance of the cooking process. For example, it can provide detailed advice on key cooking steps and simplify auxiliary steps. It can also explain the procedures for important cooking steps in detail and provide only an overview for other steps. Furthermore, it can explain the key points and precautions for key cooking steps in detail. This allows the advice unit to provide more appropriate advice by adjusting the level of detail based on the importance of the cooking process. Some or all of the above processing in the advice unit may be performed using AI, for example, or not. For example, the advice unit can input cooking importance data into a generating AI and have the generating AI perform the adjustment of the level of detail in the advice.

[0107] The advice unit can apply different advice algorithms depending on the cooking category when providing advice. For example, in the case of vegetable dishes, the advice unit provides advice based on nutritional value and cooking method. In the case of meat dishes, the advice unit can also provide advice based on the cut of meat and cooking time. Furthermore, in the case of desserts, the advice unit can provide advice based on sweetness and calories. This allows the advice unit to provide more appropriate advice by applying different advice algorithms depending on the cooking category. Some or all of the above processing in the advice unit may be performed using AI, for example, or without AI. For example, the advice unit can input cooking category data into a generating AI and have the generating AI execute the application of the advice algorithm.

[0108] The advice unit can estimate the user's emotions and adjust the length of the advice based on the estimated emotions. For example, if the user is stressed, the advice unit can provide short, concise advice. If the user is relaxed, the advice unit can provide longer advice with more detailed explanations. Furthermore, if the user is in a hurry, the advice unit can provide quick and concise advice. In this way, the advice unit can adjust the length of the advice according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the advice unit may be performed using AI or not using AI. For example, the advice unit can input user emotion data into a generative AI and have the generative AI perform the adjustment of the length of the advice.

[0109] The advice unit can prioritize advice based on the submission date of the recipe. For example, if the submission date is approaching, the advice unit will prioritize providing important advice. If the submission date is far off, the advice unit can also provide detailed advice. Furthermore, the advice unit can adjust the content and importance of the advice based on the submission date. This allows the advice unit to provide more appropriate advice by prioritizing advice based on the submission date of the recipe. Some or all of the above processing in the advice unit may be performed using AI, for example, or not using AI. For example, the advice unit can input submission date data into a generating AI and have the generating AI perform the priority determination of advice.

[0110] The advice unit can adjust the order of advice based on the relevance of the cooking process. For example, it may prioritize advice on side dishes related to the main dish. It can also sequentially provide advice on multiple cooking steps using the same ingredients. Furthermore, it can adjust the order of advice considering the interrelationships of the cooking steps. This allows the advice unit to provide more appropriate advice by adjusting the order of advice based on the relevance of the cooking process. Some or all of the above processing in the advice unit may be performed using AI, for example, or without AI. For example, the advice unit can input cooking relevance data into a generating AI and have the generating AI perform the adjustment of the order of advice.

[0111] The alternative suggestion unit can estimate the user's emotions and adjust the method of suggesting alternative methods based on the estimated emotions. For example, if the user is stressed, the alternative suggestion unit can suggest a simple and easy alternative method. If the user is relaxed, the alternative suggestion unit can also suggest an alternative method that can be enjoyed over time. Furthermore, if the user is in a hurry, the alternative suggestion unit can suggest an alternative method that can be completed in a short time. In this way, the alternative suggestion unit can adjust the method of suggesting alternative methods according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the alternative suggestion unit may be performed using AI or not using AI. For example, the alternative suggestion unit can input user emotion data into the generative AI and have the generative AI perform the adjustment of the alternative method suggestion method.

[0112] The alternative suggestion unit can adjust the level of detail of alternative methods based on the importance of the cooking process when suggesting alternative methods. For example, the alternative suggestion unit can provide detailed alternative methods for major cooking processes and simplify auxiliary processes. Alternatively, the alternative suggestion unit can provide detailed explanations of alternative methods for important cooking processes and only outlines other processes. Furthermore, the alternative suggestion unit can also provide detailed explanations of key points and considerations for alternative methods of major cooking processes. This allows the alternative suggestion unit to suggest more appropriate alternative methods by adjusting the level of detail of alternative methods based on the importance of the cooking process. Some or all of the above processing in the alternative suggestion unit may be performed using AI, for example, or not. For example, the alternative suggestion unit can input cooking importance data into a generating AI and have the generating AI perform the adjustment of the level of detail of the alternative methods.

[0113] The alternative suggestion unit can apply different alternative method algorithms depending on the cooking category when suggesting alternative methods. For example, in the case of vegetable dishes, the alternative suggestion unit can suggest alternative methods based on nutritional value and cooking method. In the case of meat dishes, the alternative suggestion unit can also suggest alternative methods based on the cut of meat and cooking time. Furthermore, in the case of desserts, the alternative suggestion unit can also suggest alternative methods based on sweetness and calories. In this way, the alternative suggestion unit can suggest more appropriate alternative methods by applying different alternative method algorithms depending on the cooking category. Some or all of the above processing in the alternative suggestion unit may be performed using AI, for example, or without AI. For example, the alternative suggestion unit can input cooking category data into a generating AI and have the generating AI execute the application of alternative method algorithms.

[0114] The alternative suggestion unit can estimate the user's emotions and adjust the length of the alternative method based on the estimated emotions. For example, if the user is stressed, the alternative suggestion unit can suggest a short, concise alternative method. If the user is relaxed, the alternative suggestion unit can suggest a longer alternative method that includes detailed explanations. Furthermore, if the user is in a hurry, the alternative suggestion unit can suggest a quick and concise alternative method. In this way, the alternative suggestion unit can adjust the length of the alternative method according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the alternative suggestion unit may be performed using AI or not using AI. For example, the alternative suggestion unit can input user emotion data into a generative AI and have the generative AI perform the length adjustment of the alternative method.

[0115] The alternative proposal unit can prioritize alternative methods based on the submission timing of the recipe when proposing alternative methods. For example, if the submission deadline is approaching, the alternative proposal unit will prioritize providing important alternative methods. If the submission deadline is far off, the alternative proposal unit can also provide detailed alternative methods. Furthermore, the alternative proposal unit can adjust the content and importance of alternative methods based on the submission timing. This allows the alternative proposal unit to propose more appropriate alternative methods by prioritizing them based on the submission timing of the recipe. Some or all of the above processing in the alternative proposal unit may be performed using AI, for example, or not using AI. For example, the alternative proposal unit can input submission timing data into a generating AI and have the generating AI perform the determination of alternative method priorities.

[0116] The alternative suggestion unit can adjust the order of alternative methods based on the relevance of the cooking process when suggesting alternative methods. For example, the alternative suggestion unit may prioritize providing alternative methods for side dishes related to the main dish. The alternative suggestion unit can also sequentially provide alternative methods for multiple cooking processes using the same ingredients. Furthermore, the alternative suggestion unit can adjust the order of alternative methods considering the interrelationships of the cooking processes. This allows the alternative suggestion unit to suggest more appropriate alternative methods by adjusting the order of alternative methods based on the relevance of the cooking process. Some or all of the above processing in the alternative suggestion unit may be performed using AI, for example, or without AI. For example, the alternative suggestion unit can input cooking relevance data into a generating AI and have the generating AI perform the order adjustment of alternative methods.

[0117] The engagement unit can estimate the user's emotions and adjust how engagement is displayed based on the estimated emotions. For example, if the user is stressed, the engagement unit can display concise and positive engagement. If the user is relaxed, the engagement unit can also display detailed engagement. Furthermore, if the user is in a hurry, the engagement unit can quickly display concise engagement. In this way, the engagement unit can adjust how engagement is displayed according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. The generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the engagement unit may be performed using AI or not using AI. For example, the engagement unit can input user emotion data into the generative AI and have the generative AI adjust how engagement is displayed.

[0118] The engagement unit can select the optimal display method by referring to the user's past operation history when displaying engagement. For example, the engagement unit can prioritize providing display methods that the user has preferred to use in the past. The engagement unit can also select the most effective display method from the user's past operation history. Furthermore, the engagement unit can provide personalized display methods based on the user's operation history. As a result, the engagement unit can enable more appropriate engagement by selecting the optimal display method by referring to the user's past operation history. Some or all of the above processing in the engagement unit may be performed using AI, for example, or without AI. For example, the engagement unit can input past operation history data into a generating AI and have the generating AI select the optimal display method.

[0119] The engagement unit can estimate the user's emotions and adjust the engagement operation procedures based on the estimated user emotions. For example, if the user is stressed, the engagement unit can provide simple and intuitive operation procedures. It can also provide detailed operation procedures if the user is relaxed. Furthermore, if the user is in a hurry, the engagement unit can provide quick and concise operation procedures. This allows the engagement unit to adjust the engagement operation procedures according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processes in the engagement unit may be performed using AI, or not. For example, the engagement unit can input user emotion data into a generative AI and have the generative AI adjust the engagement operation procedures.

[0120] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0121] The analysis unit can analyze users' food consumption patterns and make suggestions considering the expiration dates of ingredients. For example, the analysis unit can prioritize analyzing the expiration dates of ingredients that users frequently consume and suggest recipes that use ingredients that are nearing their expiration date. The analysis unit can also suggest recipes that prioritize the use of ingredients with shorter expiration dates, while delaying the use of ingredients with longer expiration dates. Furthermore, the analysis unit can analyze users' consumption patterns and make suggestions to reduce food waste. As a result, the analysis unit can make more appropriate recipe suggestions by considering users' food consumption patterns.

[0122] The suggestion function can propose recipes while taking into account the user's food allergy information. For example, if a user is allergic to a specific ingredient, the suggestion function will propose a recipe that does not include that ingredient. The suggestion function can also propose recipes that substitute ingredients that may cause allergies. Furthermore, based on the user's allergy information, the suggestion function can also propose cooking methods that avoid allergies. This allows the suggestion function to propose safer and more appropriate recipes while taking the user's allergy information into consideration.

[0123] The monitoring unit can perform monitoring while considering the user's cooking environment. For example, the monitoring unit can suggest appropriate cooking methods considering the type of equipment and cooking utensils in the user's kitchen. Furthermore, the monitoring unit can adjust cooking time and temperature according to the user's cooking environment. In addition, the monitoring unit can monitor the progress of cooking based on the user's cooking environment and provide advice at the appropriate time. This allows the monitoring unit to perform more appropriate monitoring while considering the user's cooking environment.

[0124] The feedback unit can analyze the user's cooking history and suggest areas for improvement. For example, it can point out areas for improvement based on the user's evaluation of dishes they have cooked in the past. It can also provide advice on specific cooking methods or ingredient usage based on the user's cooking history. Furthermore, it can analyze the user's cooking history and provide information useful for future cooking. This allows the feedback unit to provide more appropriate feedback by taking the user's cooking history into consideration.

[0125] The analysis unit can estimate the user's emotions and adjust the accuracy of the ingredient inventory analysis based on those emotions. For example, if the user is stressed, the analysis unit can perform a rapid analysis and immediately begin making suggestions. If the user is relaxed, the analysis unit can perform a more detailed analysis and provide more options. Furthermore, if the user is in a hurry, the analysis unit can prioritize analyzing only the most important ingredients and quickly make suggestions. In this way, the analysis unit can adjust the accuracy of the ingredient inventory analysis according to the user's emotions.

[0126] The suggestion function can estimate the user's emotions and adjust the recipe suggestions based on those emotions. For example, if the user is feeling stressed, the suggestion function can suggest a simple and easy recipe. If the user is relaxed, it can suggest a recipe that can be enjoyed at a leisurely pace. Furthermore, if the user is in a hurry, it can suggest a recipe that can be prepared in a short amount of time. In this way, the suggestion function can adjust the recipe suggestions according to the user's emotions.

[0127] The monitoring unit can estimate the user's emotions and adjust the monitoring frequency based on the estimated emotions. For example, if the user is stressed, the monitoring unit will monitor frequently and provide advice at the appropriate time. Conversely, if the user is relaxed, the monitoring unit can reduce the monitoring frequency and provide advice only when necessary. Furthermore, if the user is in a hurry, the monitoring unit can monitor only the important points and provide advice quickly. In this way, the monitoring unit can adjust the monitoring frequency according to the user's emotions.

[0128] The feedback unit can estimate the user's emotions and adjust the content of the feedback based on those emotions. For example, if the user is stressed, the feedback unit will provide concise and positive feedback. If the user is relaxed, the feedback unit can also provide detailed feedback. Furthermore, if the user is in a hurry, the feedback unit can provide concise and rapid feedback. In this way, the feedback unit can adjust the content of the feedback according to the user's emotions.

[0129] The suggestion unit can estimate the user's emotions and adjust the order of recipe suggestions based on those emotions. For example, if the user is stressed, the suggestion unit will prioritize suggesting easy and quick recipes. If the user is relaxed, the suggestion unit can also suggest recipes that can be enjoyed at a leisurely pace. Furthermore, if the user is in a hurry, the suggestion unit can quickly suggest recipes that can be prepared in a short amount of time. In this way, the suggestion unit can adjust the order of recipe suggestions according to the user's emotions.

[0130] The analysis unit can perform inventory analysis while considering the user's food storage methods. For example, the analysis unit can differentiate between food items that require refrigeration and those that can be stored at room temperature. Furthermore, the analysis unit can prioritize the analysis of food items that can be frozen and suggest recipes that allow for long-term storage. In addition, the analysis unit can offer suggestions to prevent food spoilage based on the user's storage methods. This allows the analysis unit to perform more appropriate inventory analysis by considering the user's food storage methods.

[0131] The following briefly describes the processing flow for example form 2.

[0132] Step 1: The analysis unit analyzes food inventory and purchase history. The analysis unit automatically scans the inventory in the refrigerator and pantry and saves it to the database. It can also analyze the user's purchase history to understand past purchasing patterns. For example, it can identify the food items that the user frequently purchases and update their inventory status in real time. Step 2: The suggestion unit proposes the optimal recipe based on the information analyzed by the analysis unit. The suggestion unit proposes a nutritionally balanced recipe based on the user's taste preferences and ingredient inventory. It can also propose recipes tailored to the user's cooking skill level. For example, it can suggest simple recipes for beginners and complex recipes for advanced cooks. Step 3: The monitoring unit monitors the cooking process in real time based on the recipe proposed by the suggestion unit and provides advice at the appropriate time. The monitoring unit detects the state of the ingredients during cooking using sensors and advises on the appropriate cooking time and temperature. It can also monitor the progress of the cooking procedure and notify the user of the next step. For example, it can detect when the ingredients are cooked and notify the user. Step 4: The Feedback Department provides a customized recipe based on the information monitored by the Monitoring Department, accumulating feedback for future reference. The Feedback Department collects evaluations of the dishes cooked by the user and incorporates them into future recipe suggestions. It can also provide specific advice to support the user in improving their cooking skills. For example, it can point out areas where the user needs improvement and provide information that will be useful for future cooking.

[0133] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0134] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include text generation AI, image generation AI, and multimodal generation AI. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats from audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each of the above parts is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example.Furthermore, processing performed by AI, including generative AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generative AI.

[0135] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0136] Each of the multiple elements described above, including the analysis unit, suggestion unit, monitoring unit, and feedback unit, is implemented in at least one of the smart device 14 and the data processing unit 12. For example, the analysis unit scans the inventory in the refrigerator and pantry using the camera 42 and sensors of the smart device 14, and the data is analyzed by the identification processing unit 290 of the data processing unit 12. The suggestion unit is implemented in the identification processing unit 290 of the data processing unit 12 and suggests an optimal recipe based on the user's taste preferences and ingredient inventory. The monitoring unit is implemented in the control unit 46A of the smart device 14 and monitors the cooking process in real time, providing advice at the appropriate time. The feedback unit is implemented in the identification processing unit 290 of the data processing unit 12 and provides successful experiences with customized recipes and accumulates feedback. The correspondence between each unit and the device or control unit is not limited to the example described above, and various changes are possible.

[0137] [Second Embodiment] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0138] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0139] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0140] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0141] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0142] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0143] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0144] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing by the processor 28. The storage 32 stores the specific processing program 56.

[0145] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0146] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0147] In the smart glasses 214, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0148] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0149] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0150] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0151] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0152] Each of the multiple elements described above, including the analysis unit, suggestion unit, monitoring unit, and feedback unit, is implemented, for example, in at least one of the smart glasses 214 and the data processing unit 12. For example, the analysis unit uses the camera 42 and sensors of the smart glasses 214 to scan the inventory in the refrigerator and pantry, and the data is analyzed by the identification processing unit 290 of the data processing unit 12. The suggestion unit is implemented, for example, by the identification processing unit 290 of the data processing unit 12, and suggests an optimal recipe based on the user's taste preferences and ingredient inventory. The monitoring unit is implemented, for example, by the control unit 46A of the smart glasses 214, and monitors the cooking process in real time and provides advice at the appropriate time. The feedback unit is implemented, for example, by the identification processing unit 290 of the data processing unit 12, and provides successful experiences with customized recipes and accumulates feedback. The correspondence between each unit and the device or control unit is not limited to the example described above, and various changes are possible.

[0153] [Third Embodiment] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0154] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0155] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0156] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0157] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0158] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0159] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0160] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0161] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0162] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0163] In the headset terminal 314, specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50 and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific program 60 executed on the RAM 48. The headset terminal 314 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0164] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0165] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0166] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0167] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0168] Each of the multiple elements described above, including the analysis unit, suggestion unit, monitoring unit, and feedback unit, is implemented in at least one of the headset terminal 314 and the data processing unit 12. For example, the analysis unit scans the inventory in the refrigerator and pantry using the camera 42 and sensors of the headset terminal 314, and the data is analyzed by the identification processing unit 290 of the data processing unit 12. The suggestion unit is implemented in the identification processing unit 290 of the data processing unit 12 and suggests an optimal recipe based on the user's taste preferences and ingredient inventory. The monitoring unit is implemented in the control unit 46A of the headset terminal 314 and monitors the cooking process in real time, providing advice at the appropriate time. The feedback unit is implemented in the identification processing unit 290 of the data processing unit 12 and provides successful experiences with customized recipes and accumulates feedback. The correspondence between each unit and the device or control unit is not limited to the example described above, and various changes are possible.

[0169] [Fourth Embodiment] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0170] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0171] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0172] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0173] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0174] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS image sensor or CCD image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0175] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0176] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. The robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0177] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0178] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0179] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0180] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in storage 50. Processor 46 reads the specific program 60 from storage 50 and executes it on RAM 48. The specific processing is achieved by processor 46 acting as a control unit 46A according to the specific program 60 executed on RAM 48. Robot 414 also has data generation model 58 and emotion identification model 59, similar to those of the robot, and can perform processing similar to that of the specific processing unit 290 using these models.

[0181] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0182] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0183] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0184] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the robot 414 or an external device, and the robot 414 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0185] Each of the multiple elements described above, including the analysis unit, suggestion unit, monitoring unit, and feedback unit, is implemented in at least one of the robot 414 and the data processing unit 12. For example, the analysis unit uses the camera 42 and sensors of the robot 414 to scan the inventory in the refrigerator and pantry, and the data is analyzed by the identification processing unit 290 of the data processing unit 12. The suggestion unit is implemented in the identification processing unit 290 of the data processing unit 12 and suggests an optimal recipe based on the user's taste preferences and ingredient inventory. The monitoring unit is implemented in the control unit 46A of the robot 414 and monitors the cooking process in real time, providing advice at the appropriate time. The feedback unit is implemented in the identification processing unit 290 of the data processing unit 12 and provides successful experiences with customized recipes and accumulates feedback. The correspondence between each unit and the device or control unit is not limited to the example described above, and various changes are possible.

[0186] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0187] Figure 9 shows the emotion map 400, in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0188] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0189] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0190] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, and motorcycles, emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0191] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0192] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0193] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0194] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0195] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0196] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0197] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0198] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0199] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0200] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0201] Furthermore, although the above-described examples were divided into four embodiments, some or all of these embodiments may be combined. Also, the smart device 14, smart glasses 214, headset terminal 314, and robot 414 are just examples, and they may be combined, or other devices may be used. Also, although the above-described examples were divided into two embodiments, Embodiment 1 and Embodiment 2, these may be combined.

[0202] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and other things that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0203] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0204] (Note 1) The analysis unit analyzes the inventory and purchase history of ingredients, A proposal unit that proposes the optimal recipe based on the information analyzed by the aforementioned analysis unit, A monitoring unit monitors the cooking process in real time based on the recipe proposed by the aforementioned proposal unit and provides advice at the appropriate time. The system includes a feedback unit that provides a customized recipe for successful experiences based on information monitored by the monitoring unit, and stores feedback for future reference. A system characterized by the following features. (Note 2) It includes an advice department that provides advice tailored to the user's taste preferences. The system described in Appendix 1, characterized by the features described herein. (Note 3) It includes an alternative proposal unit that suggests alternative methods as needed. The system described in Appendix 1, characterized by the features described herein. (Note 4) It features an engagement unit that promotes user engagement through real-time feedback. The system described in Appendix 1, characterized by the features described herein. (Note 5) The aforementioned analysis unit, The system estimates user sentiment and adjusts the timing of ingredient inventory analysis based on the estimated user sentiment. The system described in Appendix 1, characterized by the features described herein. (Note 6) The aforementioned analysis unit, Analyze the user's past purchase history and select the optimal analysis method. The system described in Appendix 1, characterized by the features described herein. (Note 7) The aforementioned analysis unit, When analyzing food inventory, filtering is performed based on the user's current health status and dietary restrictions. The system described in Appendix 1, characterized by the features described herein. (Note 8) The aforementioned analysis unit, It estimates the user's emotions and determines the priority of ingredients to analyze based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 9) The aforementioned analysis unit, When analyzing food inventory, the system prioritizes analyzing highly relevant ingredients by considering the user's geographical location. The system described in Appendix 1, characterized by the features described herein. (Note 10) The aforementioned analysis unit, When analyzing food inventory, we analyze users' social media activity and identify related food items. The system described in Appendix 1, characterized by the features described herein. (Note 11) The aforementioned proposal section is, The system estimates the user's emotions and adjusts the recipe suggestion method based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 12) The aforementioned proposal section is, When suggesting recipes, adjust the level of detail based on the importance of the ingredients. The system described in Appendix 1, characterized by the features described herein. (Note 13) The aforementioned proposal section is, When suggesting recipes, different suggestion algorithms are applied depending on the category of ingredients. The system described in Appendix 1, characterized by the features described herein. (Note 14) The aforementioned proposal section is, It estimates the user's emotions and adjusts the length of the suggestion based on the estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 15) The aforementioned proposal section is, When suggesting recipes, we prioritize suggestions based on the freshness of the ingredients. The system described in Appendix 1, characterized by the features described herein. (Note 16) The aforementioned proposal section is, When suggesting recipes, adjust the order of suggestions based on the relationships between ingredients. The system described in Appendix 1, characterized by the features described herein. (Note 17) The monitoring unit, The system estimates user sentiment and adjusts monitoring criteria based on the estimated user sentiment. The system described in Appendix 1, characterized by the features described herein. (Note 18) The monitoring unit, During monitoring, consider the interrelationships between cooking processes to improve monitoring accuracy. The system described in Appendix 1, characterized by the features described herein. (Note 19) The monitoring unit, During monitoring, the monitor will take into account the attributes of the cook. The system described in Appendix 1, characterized by the features described herein. (Note 20) The monitoring unit, It estimates the user's emotions and adjusts the order in which monitoring results are displayed based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 21) The monitoring unit, During monitoring, the geographical distribution of cooking locations should be taken into consideration. The system described in Appendix 1, characterized by the features described herein. (Note 22) The monitoring unit, During monitoring, we refer to relevant literature on cooking to improve the accuracy of the monitoring. The system described in Appendix 1, characterized by the features described herein. (Note 23) The aforementioned feedback unit is It estimates the user's emotions and adjusts how feedback is displayed based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 24) The aforementioned feedback unit is When providing feedback, past feedback data is used to predict current feedback. The system described in Appendix 1, characterized by the features described herein. (Note 25) The aforementioned feedback unit is When providing feedback, different feedback analysis methods are applied to each cooking category. The system described in Appendix 1, characterized by the features described herein. (Note 26) The aforementioned feedback unit is It estimates the user's emotions and adjusts the importance of feedback based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 27) The aforementioned feedback unit is When providing feedback, we analyze how the feedback changes based on when the cooking was submitted. The system described in Appendix 1, characterized by the features described herein. (Note 28) The aforementioned feedback unit is When providing feedback, we analyze the feedback by referring to relevant market data for cooking. The system described in Appendix 1, characterized by the features described herein. (Note 29) The aforementioned advice section, It estimates the user's emotions and adjusts the way advice is presented based on those estimated emotions. The system described in Appendix 2, characterized by the features described herein. (Note 30) The aforementioned advice section, When giving advice, adjust the level of detail based on the importance of the dish. The system described in Appendix 2, characterized by the features described herein. (Note 31) The aforementioned advice section, When providing advice, different advice algorithms are applied depending on the cooking category. The system described in Appendix 2, characterized by the features described herein. (Note 32) The aforementioned advice section, It estimates the user's emotions and adjusts the length of the advice based on those emotions. The system described in Appendix 2, characterized by the features described herein. (Note 33) The aforementioned advice section, When giving advice, we prioritize the advice based on when the cooking is submitted. The system described in Appendix 2, characterized by the features described herein. (Note 34) The aforementioned advice section, When giving advice, adjust the order of advice based on its relevance in cooking. The system described in Appendix 2, characterized by the features described herein. (Note 35) The aforementioned alternative proposal unit is, It estimates the user's emotions and adjusts the proposed alternative method based on the estimated user emotions. The system described in Appendix 3, characterized by the features described herein. (Note 36) The aforementioned alternative proposal unit is, When proposing alternative methods, adjust the level of detail of the alternative methods based on the importance of the cooking process. The system described in Appendix 3, characterized by the features described herein. (Note 37) The aforementioned alternative proposal unit is, When proposing alternative methods, different alternative method algorithms are applied depending on the cooking category. The system described in Appendix 3, characterized by the features described herein. (Note 38) The aforementioned alternative proposal unit is, The system estimates the user's emotions and adjusts the length of the alternative method based on the estimated user emotions. The system described in Appendix 3, characterized by the features described herein. (Note 39) The aforementioned alternative proposal unit is, When proposing alternative methods, prioritize them based on the submission timing of the cooking process. The system described in Appendix 3, characterized by the features described herein. (Note 40) The aforementioned alternative proposal unit is, When proposing alternative methods, adjust the order of the alternative methods based on their relevance in cooking. The system described in Appendix 3, characterized by the features described herein. (Note 41) The aforementioned engagement unit is, It estimates user sentiment and adjusts how engagement is displayed based on the estimated user sentiment. The system described in Appendix 4, characterized by the features described herein. (Note 42) The aforementioned engagement unit is, When displaying engagement, the system selects the optimal display method by referring to the user's past activity history. The system described in Appendix 4, characterized by the features described herein. (Note 43) The aforementioned engagement unit is, It estimates the user's emotions and adjusts the engagement process based on those emotions. The system described in Appendix 4, characterized by the features described herein. (Note 44) The aforementioned engagement unit is, When displaying engagement data, the optimal display method is selected considering the user's device information. The system described in Appendix 4, characterized by the features described herein. [Explanation of symbols]

[0205] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots

Claims

1. The analysis unit analyzes the inventory and purchase history of ingredients, A proposal unit that proposes the optimal recipe based on the information analyzed by the aforementioned analysis unit, A monitoring unit monitors the cooking process in real time based on the recipe proposed by the aforementioned proposal unit and provides advice at the appropriate time. The system includes a feedback unit that provides a customized recipe for successful experiences based on information monitored by the monitoring unit, and stores feedback for future reference. A system characterized by the following features.

2. It includes an advice department that provides advice tailored to the user's taste preferences. The system according to feature 1.

3. It includes an alternative proposal unit that suggests alternative methods as needed. The system according to feature 1.

4. It features an engagement unit that promotes user engagement through real-time feedback. The system according to feature 1.

5. The aforementioned analysis unit, The system estimates user sentiment and adjusts the timing of ingredient inventory analysis based on the estimated user sentiment. The system according to feature 1.

6. The aforementioned analysis unit, Analyze the user's past purchase history and select the optimal analysis method. The system according to feature 1.

7. The aforementioned analysis unit, When analyzing food inventory, filtering is performed based on the user's current health status and dietary restrictions. The system according to feature 1.

8. The aforementioned analysis unit, It estimates the user's emotions and determines the priority of ingredients to analyze based on the estimated user emotions. The system according to feature 1.

Images