Control data generation device, control data generation method, control data generation program, control data generation system, creation method, trained model, and adjusted model

Abstract

The present invention addresses the problem of providing a control data generation device and the like that contribute to controlling, in real time, a cooking process (for example, frying, stirring, and / or additionally inputting oil or seasoning) being executed on the basis of recipe data, according to sound during cooking and the appearance of a material (for example, an ingredient) being cooked. A control data generation device according to the present embodiment: generates control data for controlling a cooking process on the basis of sound measurement data which pertains to sound during execution of a cooking process based on recipe data (the recipe data including cooking process information) and which has been obtained by measurement using a sensor, appearance measurement data which pertains to the appearance of a material being cooked during the execution and which has been obtained by measurement using a sensor, and the recipe data; and outputs the generated control data.

Description

Control data generation device, control data generation method, control data generation program, control data generation system, creation method, learned model, and adjusted model

[0001] The present invention relates to a control data generation device, a control data generation method, a control data generation program, a control data generation system, a creation method, a learned model, and an adjusted model.

[0002] Patent Document 1 discloses an information processing device and a cooking system that can present more sensory information sensed at a remote location to a user operating a robot at the remote location. Patent Document 2 discloses a system and method for conveniently preparing a chef's gourmet cuisine at a consumer's home and serving it to the consumer without the need to visit each restaurant around the world to enjoy a specific gourmet dish. Patent Document 3 discloses a technique for enabling the generation of new recipes. Patent Document 4 discloses a technique for enabling the easy measurement of the aroma of food ingredients to be cooked. Patent Document 5 discloses a technique for enabling the customization of dishes made by a cooking robot according to the preferences of the person eating them. Patent Document 6 discloses a technique for enhancing the reproducibility when reproducing the same dish as that made by a cook in a cooking robot.

[0003] International Publication No. 2022 / 102287, Japanese Unexamined Patent Application Publication No. 2022 - 101582, International Publication No. 2021 / 024829, International Publication No. 2021 / 024828, International Publication No. 2020 / 179402, International Publication No. 2020 / 179401

[0004] However, in the prior art, it is not possible to control, in real time, a cooking process in progress (for example, frying, stirring, adding additional oil or seasonings, etc.) based on recipe data according to the sound during cooking and the appearance of the food being cooked (for example, food ingredients, etc.).

[0005] The present invention has been made in view of the above problems, and aims to provide a control data generation device, a control data generation method, a control data generation program, and a control data generation system that contribute to real-time control of a cooking process in progress based on recipe data, in accordance with the sounds during cooking and the appearance of the food being cooked, as well as a method for creating a trained model that generates control data and a tuned model that performs the task of generating control data, which contribute to the real-time control, and a trained model and a tuned model.

[0006] To solve the above-mentioned problems and achieve the objective, the control data generation device according to the present invention is characterized by comprising: a generation unit that can access a storage unit that stores recipe data including cooking process information, and generates control data for controlling a cooking process based on measurement data obtained by sensors, which includes sound measurement data relating to the sound during the execution of a certain cooking process based on the recipe data, appearance measurement data relating to the appearance of the food being cooked during the execution, and the recipe data; and an output processing unit that outputs the control data generated by the generation unit.

[0007] The output processing unit may also output the control data to a drive unit that drives the cooking appliance.

[0008] Furthermore, the output processing unit may output the control data to the user.

[0009] Furthermore, the output processing unit may output the control data to the drive unit that operates the cooking appliance and to the user.

[0010] Furthermore, the control data generation device according to the present invention may further include an acquisition unit that acquires the recipe data and stores it in the storage unit, and the generation unit may generate the control data based on the sound measurement data, the appearance measurement data and the recipe data acquired by the acquisition unit.

[0011] Furthermore, the recipe data may include information created based on reinforcement learning.

[0012] Furthermore, the acquisition unit may acquire user data including information created based on reinforcement learning and store it further in the storage unit; the sound measurement data may be measurement data relating to the sound during the execution of a certain cooking process based on the recipe data and the user data; the appearance measurement data may be measurement data relating to the appearance of the cooked object during the execution; and the generation unit may generate the control data based on the sound measurement data, the appearance measurement data, the recipe data acquired by the acquisition unit, and the user data acquired by the acquisition unit.

[0013] Furthermore, the control data generation device according to the present invention may further include the storage unit, and the storage unit may further store the measurement data.

[0014] Furthermore, the measurement data may include information created based on reinforcement learning.

[0015] Furthermore, the control data may be used to control the start of cooking, the end of cooking, heating conditions, stirring conditions, or any combination thereof.

[0016] Furthermore, the generation unit may generate the control data based on (1) the sound measurement data, the appearance measurement data, the recipe data, and the user data, and (2) the resistance measurement data relating to the resistance value of the cooking appliance in operation, the odor measurement data relating to the odor components of the food being cooked in operation, the taste measurement data relating to the taste components of the food being cooked in operation, the temperature measurement data relating to the temperature of the food being cooked in operation, the weight measurement data relating to the weight of the food being cooked in operation, the elasticity measurement data relating to the elasticity of the food being cooked in operation, or any combination thereof.

[0017] Furthermore, the cooking process information included in the recipe data may be detailed by a generating AI that has been given a prompt to further detail the cooking process information, and the prompt may include an explanation of the concept of the detailing and may also include cooking utensil data describing the cooking utensils used for each ingredient and preparation process, and usage data describing the uses of each cooking utensil, as auxiliary information for the detailing.

[0018] Furthermore, the control data generation method according to the present invention is characterized in that an information processing device that can access a storage unit that stores recipe data including cooking process information generates control data for controlling a cooking process based on measurement data obtained by sensors, which includes sound measurement data relating to the sound during the execution of a certain cooking process based on the recipe data, appearance measurement data relating to the appearance of the food being cooked during the execution, and the recipe data, and outputs the control data generated in the generation step.

[0019] Furthermore, the control data generation program according to the present invention causes an information processing device that can access a storage unit that stores recipe data including cooking process information to execute a generation step of generating control data for controlling a cooking process based on measurement data obtained by sensors, which includes sound measurement data relating to the sound during the execution of a certain cooking process based on the recipe data, appearance measurement data relating to the appearance of the food being cooked during the execution, and the recipe data; and an output processing step of outputting the control data generated in the generation step.

[0020] Furthermore, the recording medium according to the present invention is a non-temporary, computer-readable recording medium that records the control data generation program. In other words, the recording medium according to the present invention is a non-temporary, computer-readable recording medium characterized by including programmed instructions for causing a computer to execute the control data generation method.

[0021] Furthermore, the control data generation system according to the present invention is characterized by comprising: a sound sensor capable of measuring sound; an appearance sensor capable of measuring the appearance of an object; a storage unit that stores recipe data including cooking process information; a generation unit that generates control data for controlling a cooking process based on sound measurement data related to the sound during the execution of a certain cooking process based on the recipe data, obtained by measuring with the sound sensor; appearance measurement data related to the appearance of the object being cooked during the execution, obtained by measuring with the appearance sensor; and the recipe data; and an output processing unit that outputs the control data generated by the generation unit.

[0022] Furthermore, the control data generation system according to the present invention may further include a cooking appliance and a drive unit for driving the cooking appliance, and the output processing unit may output the control data to the drive unit.

[0023] Furthermore, the output processing unit may output the control data to the user.

[0024] Furthermore, the control data generation system according to the present invention may further include a cooking appliance and a drive unit for driving the cooking appliance, and the output processing unit may output the control data to the drive unit and the user.

[0025] Furthermore, the present invention is characterized by creating a trained model by using training data which associates sound measurement data related to the sound during execution of a cooking process based on recipe data that includes cooking process information, which is measurement data obtained by measuring with a sensor, appearance measurement data which is measurement data related to the appearance of the food being cooked during execution, and recipe data with control data for controlling the cooking process, and by performing a parameter training process for a machine learning model that outputs control data when sound measurement data, appearance measurement data and recipe data are input.

[0026] Furthermore, the creation method according to the present invention is characterized by creating an adjusted model that performs the task of generating control data for controlling a cooking process by using a dataset that associates sound measurement data relating to the sound during the execution of a certain cooking process based on recipe data including cooking process information, appearance measurement data relating to the appearance of the food being cooked during the execution, and recipe data with control data for controlling the cooking process, and performing parameter adjustment processing of the base model.

[0027] Furthermore, the trained model according to the present invention is a trained model that outputs control data for controlling a cooking process when sound measurement data relating to the sound during the execution of a cooking process based on recipe data including cooking process information, which is measurement data obtained by measurement data measured by a sensor, appearance measurement data relating to the appearance of the food being cooked during the execution, and recipe data are input, the parameters of the trained model are learned using training data that associates sound measurement data, appearance measurement data, and recipe data with control data, and the computer is configured to accept sound measurement data, appearance measurement data, and recipe data as input, perform calculations based on the parameters on the input sound measurement data, appearance measurement data, and recipe data, and output control data.

[0028] Furthermore, the adjusted model according to the present invention is an adjusted model that, upon input of measurement data obtained by measurement with a sensor, including sound measurement data relating to the sound during the execution of a certain cooking process based on recipe data including cooking process information, appearance measurement data relating to the appearance of the food being cooked during the execution, and recipe data, performs the task of generating control data for controlling the cooking process. The parameters of the adjusted model are parameters of the base model, adjusted using a dataset that associates sound measurement data, appearance measurement data, and recipe data with control data. The computer is configured to accept sound measurement data, appearance measurement data, and recipe data as input, perform calculations based on the parameters on the input sound measurement data, appearance measurement data, and recipe data, and perform the task of generating control data.

[0029] Furthermore, the control data generation method according to the present invention is characterized in that an information processing device that can access a storage unit that stores recipe data including cooking process information performs a generation step of generating control data for controlling a cooking process using a machine learning model based on measurement data obtained by sensors, which includes sound measurement data relating to the sound during the execution of a certain cooking process based on the recipe data, appearance measurement data relating to the appearance of the food being cooked during the execution, and the recipe data, and an output processing step of outputting the control data generated in the generation step, wherein the machine learning model is a trained model created by the creation method described above.

[0030] Furthermore, the control data generation method according to the present invention is characterized in that an information processing device that can access a storage unit that stores recipe data including cooking process information performs a generation step of generating control data for controlling a cooking process using a base model based on measurement data obtained by sensors, which is sound measurement data relating to the sound during the execution of a certain cooking process based on the recipe data, appearance measurement data relating to the appearance of the food being cooked during the execution, and the recipe data, and an output processing step of outputting the control data generated in the generation step, wherein the base model is an adjusted model created by the creation method.

[0031] The present invention contributes to the real-time control of cooking processes in progress (e.g., stir-frying, stirring, adding oil or seasonings, etc.) based on recipe data, in accordance with the sounds during cooking and the appearance of the food being cooked (e.g., ingredients). Furthermore, the present invention has the effect of enabling the creation of a trained model that generates control data and a tuned model that performs the task of generating control data, which contribute to this real-time control, or providing a trained model and a tuned model that contribute to this real-time control.

[0032] Figure 1 is a block diagram showing the configuration of the control data generation system 1. Figure 2 is a diagram showing the processes executed by the control data generation system 1. Figure 3 is a conceptual diagram showing the process of creating and training a model and providing the trained model. Figure 4 is a conceptual diagram showing the process of generating control data using the adjusted model. Figure 5 is a diagram showing an example of a cooking utensil list and a usage list. Figure 6 is a diagram showing the relationship between the four concepts defined when refining a recipe: "recipe process," "task," "module," and "option." Figure 7 is a diagram showing the relationship between the four concepts defined when refining a recipe: "task," "module," "option," and "detailed process." Figure 8 is a diagram showing a specific example of a module. Figure 9 is a diagram showing an example of the recipe refinement process. Figure 10 is a diagram showing an example of the recipe refinement process and the recipe refinement result evaluation process. Figure 11 is a diagram showing an example of the recipe refinement process and the image generation process.

[0033] The following describes in detail, with reference to the drawings, embodiments of the control data generation device, control data generation method, control data generation program, control data generation system, creation method, trained model, and adjusted model according to the present invention. However, the present invention is not limited to these embodiments.

[0034] [1. Overview] This embodiment relates to an algorithm for reproducing cooking (e.g., heating) using ingredients (e.g., ingredients that can be purchased at supermarkets, etc.) at home using a household cooking robot (e.g., an automatic cooking appliance, etc.) or in a food factory using a commercial cooking robot.

[0035] For example, when performing cooking that simulates a stir-frying process, existing methods (such as TechMagic's cooking robot "I-Robo") require setting cooking conditions such as temperature, time, and stirring speed at the start of cooking, and furthermore, extensive prior experiments are necessary to determine these cooking conditions. In addition, if the combination, quantity, or state of ingredients changes, prior experiments must be conducted for each pattern of ingredient combination, quantity, or state to determine the cooking conditions, and then the settings must be changed to the determined cooking conditions. Moreover, since the end state of the cooking conditions (for example, the final temperature to be reached) cannot be determined at the start of cooking, it is necessary to determine the end state of the cooking conditions through prior experiments and then determine the cooking conditions necessary to achieve that determined state.

[0036] Incidentally, during cooking, a chef constantly monitors the appearance of the ingredients, the resistance felt in their hands while mixing, the sounds, smells, and changes in the taste of the ingredients using their five senses (sight, touch, hearing, smell, and taste), and adjusts the heat and stirring method as needed to complete the dish.

[0037] Therefore, in order to mimic the actions of a chef during cooking, in this embodiment, cooking endpoint information regarding the appearance, resistance, sound, smell, and taste of the finished dish is set in advance. Then, during cooking, various sensors are used to measure in real time, and the following data is analyzed in an integrated manner: appearance measurement data (using a visual sensor), resistance measurement data of the mixing jig (using a pressure sensor), sound measurement data (using a microphone), olfactory measurement data (using an olfactory sensor), taste measurement data (using a taste sensor), and food temperature measurement data (using a thermograph). This allows the cooking conditions of the cooking equipment (cooking robot), such as heating conditions and stirring conditions, to be adjusted in real time while cooking is performed.

[0038] Furthermore, according to this embodiment, even if the combination, quantity, or state of ingredients, cooking equipment, cooking utensils, or cooking environment (e.g., air temperature) changes, cooking conditions such as adjusting the heat can be adjusted in real time based on data measured in real time, thus eliminating the need for prior experiments. In addition, this embodiment can be used not only to generate control data for cooking robots, but also to present cooking methods to humans (for example, providing information on cooking methods to a person wearing sensors while cooking). Specifically, this embodiment can be used not only to output the generated control data to a cooking robot, but also to output the generated control data to a human in a manner that can assist the human in cooking. More specifically, this embodiment enables not only (1) "automatic cooking by a cooking robot" in which a computer transmits cooking instructions to a cooking robot using an implemented algorithm, and the cooking robot cooks completely automatically, but also (2) "manual cooking by a human" in which a computer transmits cooking instructions to a human using an implemented algorithm, and the human cooks manually, and (3) "hybrid cooking by a robot and a human" in which a computer transmits cooking instructions to a robot and a human using an implemented algorithm, and the robot and human cooperate, with the robot cooking automatically and the human cooking manually.

[0039] [2. Configuration and Processing] Figure 1 is a block diagram showing the configuration of the control data generation system 1. Figure 2 is a diagram showing the processing performed by the control data generation system 1.

[0040] The control data generation system 1 according to this embodiment consists of a control data generation device 100, a sensor 200, and a drive unit 300 for driving a cooking robot. Alternatively, the control data generation system 1 may consist of only the control data generation device 100 and the sensor 200.

[0041] The control data generation device 100 generates control data for controlling the cooking process based on recipe data including measurement data obtained by measuring with the sensor 200 and cooking process information, and outputs the generated control data to the drive unit 300 and / or the user.

[0042] The control data generation device 100 includes: (1) a control unit 102 that comprehensively controls the device such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit); (2) a communication interface unit 104 that communicably connects the device to a network via a communication device such as a router and a wired or wireless communication line such as a dedicated line; (3) a storage unit 106 that stores various databases, tables, files, etc.; and (4) an input / output interface unit 108 that connects to an input device 112 and an output device 114. Each unit included in the control data generation device 100 is communicably connected via an arbitrary communication path.

[0043] The control unit 102 has an internal memory for storing control programs such as an OS, programs defining various processing procedures, required data, etc., and executes various information processes based on these programs.

[0044] The communication interface unit 104 mediates communication between the control data generation device 100 and a network (or a communication device such as a router). That is, the communication interface unit 104 has a function of communicating data with the sensor 200 and the drive unit 300 via a communication line.

[0045] The storage unit 106 is a storage means. As the storage unit 106, for example, a memory device such as a RAM or a ROM, a fixed disk device such as a hard disk, a flexible disk, or an optical disk can be used. A computer program for giving commands to the CPU or GPU to perform various processes in cooperation with the OS (Operating System) may be recorded in the storage unit 106. When a server (not shown) is communicably connected to the control data generation device 100, the data stored in the storage unit 106 may be stored in the server.

[0046] An input / output interface unit 108 is connected to an input device 112 and an output device 114. As the output device 114, in addition to a monitor, a speaker or a printer can be used. As the input device 112, in addition to a keyboard, a mouse or a microphone, a monitor that realizes a pointing device function in cooperation with the mouse, or a touch panel can be used.

[0047] Functionally conceptually, the control unit 102 includes an acquisition unit 102a (corresponding to the acquisition unit of the present invention), a generation unit 102b (corresponding to the generation unit of the present invention), an output processing unit 102c (corresponding to the output processing unit of the present invention), and the like. Note that the control unit 102 only needs to include at least the generation unit 102b and the output processing unit 102c.

[0048] The acquisition unit 102a acquires user data including the user's attribute information (such as age, gender, place of origin, etc.) and the user's preference information regarding food, and recipe data regarding a recipe of a dish including cooking process information, and stores it in the storage unit 106a. The acquisition unit 102a may acquire data through reinforcement learning. For example, the acquisition unit 102a may acquire data by using the taste, smell or texture of the cooked dish as the target variable and learning various measurement data described later obtained during the execution of the cooking process. The user data may include information created based on reinforcement learning (for example, (1) the user's cooking skill, (2) the taste, smell or texture of a dish that the user is assumed to like or dislike, or (3) the cooking method that the user is assumed to like or dislike, etc.). The recipe data may include information created based on reinforcement learning (for example, a cooking method (such as the start of cooking, the end of cooking, heating conditions or stirring conditions, etc.) adjusted (optimized) to match the user's preference information (such as the taste, smell or texture of a dish), or a cooking method adjusted (optimized) to match the user's preference information (such as the taste, smell or texture of a dish) according to the combination, amount or state of ingredients, cooking equipment, cooking utensils or cooking environment (such as temperature or temperature, etc.), etc.).

[0049] The generation unit 102b uses the sound measurement data, appearance measurement data, and recipe data described below (which may include, for example, information regarding a standard or typical "execution time for the cooking process") to generate control data for controlling the cooking process (e.g., the start of cooking, the end of cooking, heating conditions, or stirring conditions (e.g., stirring speed, stirring frequency, etc.)) using a rule-based AI (artificial intelligence) or a specialized AI such as a machine learning model, or a general-purpose AI such as a base model (e.g., a multimodal base model). Specifically, it generates control data for causing the drive unit 300 that drives the cooking robot to control the cooking process, control data for causing the user to control the cooking process, or control data for causing the drive unit 300 and the user to cooperate in controlling the cooking process. - Sound measurement data related to the sounds during a cooking process (cooking) based on "recipe data" or "recipe data and user data," obtained by measuring with a sensor that obtains auditory information (specifically, a microphone). - Appearance measurement data related to the appearance of the food being cooked (e.g., ingredients) during cooking, obtained by measuring with a sensor that obtains visual information (specifically, a visual sensor) (e.g., appearance measurement data related to the movement of the food being cooked during cooking).

[0050] Furthermore, the generation unit 102b may generate control data using one or more measurement data selected from the following list of measurement data: • Resistance measurement data related to the resistance of the mixing jig during cooking, obtained by measuring with a sensor that obtains tactile information (specifically, a pressure sensor) • Odor measurement data related to the odor components of the food being cooked, obtained by measuring with a sensor that obtains olfactory information (specifically, an olfactory sensor) • Taste measurement data related to the taste components of the food being cooked, obtained by measuring with a sensor that obtains gustatory information (specifically, a gustatory sensor) • Temperature measurement data related to the temperature of the food being cooked, obtained by measuring with a sensor that obtains temperature information (specifically, a thermograph) • Weight measurement data related to the weight of the food being cooked, obtained by measuring with a sensor that obtains weight information (specifically, a weight sensor) • Elasticity measurement data related to the elasticity of the food being cooked, obtained by measuring with a sensor that obtains elasticity information (specifically, a texture sensor)

[0051] Furthermore, the generation unit 102b may generate control data using user data.

[0052] Furthermore, the various measurement data mentioned above may include information created based on reinforcement learning. For example, the movement of food being cooked when a cooking utensil (such as a pot) is moved can change in accordance with the change in viscosity of the oil during cooking due to temperature changes. Therefore, visual measurement data regarding the movement of food being cooked during cooking may be learned together with temperature information obtained by sensors. Specifically, when the oil temperature is low, the viscosity of the oil is high, making it difficult for food being cooked to move, but when the oil temperature is high, the viscosity of the oil is low, making it easier for food being cooked to move. Therefore, visual measurement data regarding the movement of food being cooked during cooking may be learned together with temperature information obtained by sensors.

[0053] Furthermore, rule-based AI can be constructed by the designers defining their own conditions based on the various measurement data, recipe data, and user data mentioned above. For example, the generation unit 102b may generate control data to terminate the stir-frying process when the conditions are met, such as "the temperature of the food being cooked is above a predetermined value, it is glossy, and the resistance value when the food being cooked slides on the surface of the frying pan is a predetermined value." Alternatively, for example, the generation unit 102b may generate control data to start stirring when the conditions are met, such as "the wavelength and amplitude of the sound during cooking are both predetermined values." The conditions used in the rule-based AI may be stored in the memory unit 106.

[0054] Furthermore, the machine learning model can be constructed by learning the relationship between the various measurement data, recipe data, and user data and the cooking process. For example, a model may be obtained based on reinforcement learning targeting a desirable taste, smell, or texture. For example, a learning process may be introduced so that the cooking process is optimized to better suit the user's preferences as the control data generation device 100 is used. The control data generation device 100 (or its control unit 102) may further include a first creation unit (not shown) that creates a trained machine learning model (trained model). The first creation unit may create a trained model by using the learning data, which associates the various measurement data, recipe data, and user data with the control data, to perform a parameter learning process for a machine learning model that outputs control data when the various measurement data, recipe data, and user data are input. The generation unit 102b may generate control data using the created trained model. For the process of creating and training the machine learning model, please refer to Figure 3. As shown in Figure 3, the trained model may be provided (for a fee) to the cooking robot development company. Furthermore, the control data generation device 100 may remotely control multiple cooking robots using the trained model. The first creation unit may create a trained model (specifically, an optimized model) suitable for each of the many different types of dishes. The first creation unit may (1) create a trained model (a basic model) that can skillfully cook any type of dish, and (2) use a dedicated application (containing information on unique cooking techniques) prepared for each type of dish, based on the created trained model, to create a trained model (an advanced model) suitable for each type of dish. The dedicated application makes it possible to apply to rare and unusual dishes as well. A unique cooking technique is a technique that is necessary for a particular type of dish but does not appear in other types of dishes. For example, in the case of Italian pasta dishes, the technique of emulsifying in a frying pan is a unique cooking technique that does not appear in any other type of dish.

[0055] Furthermore, the control data generation device 100 (specifically the control unit 102) may further include a second creation unit (not shown) that creates an adjusted base model (adjusted model) that performs the task of generating control data. The second creation unit may create an adjusted model (multimodal generation AI) by performing parameter adjustment processing (specifically fine tuning) of the base model using a dataset that associates the above-mentioned various measurement data, recipe data, and user data with the control data. For the process of generating control data using the adjusted model, please refer to Figure 4. The data types of the multimodal information may be determined, for example, based on the results of an analysis of which sensory information the chef uses to finish the dish. Alternatively, a multimodal generation AI may be created that analyzes the determined multimodal information and generates control data. The second creation unit may create an adjusted model (specifically an optimized model) suitable for each of the many different types of dishes. The second creation department may (1) create a pre-adjusted model (a basic model) that can skillfully prepare any type of dish, and (2) use a dedicated app (containing information on unique cooking techniques) prepared for each type of dish, based on the created pre-adjusted model, to create a pre-adjusted model (an advanced model) suitable for that type of dish. The dedicated app makes it possible to apply the model to rare or unusual dishes as well. A unique cooking technique is a technique that is necessary for a particular type of dish but does not appear in other types of dishes. For example, in the case of Italian pasta dishes, the technique of emulsifying the batter in a frying pan is a unique cooking technique that does not appear in any other type of dish.

[0056] The output processing unit 102c outputs the control data generated by the generation unit 102b to the drive unit 300, which drives the cooking robot. When the drive unit 300 receives the control data, it drives the cooking robot (specifically, the heating mechanism or stirring mechanism that constitutes the cooking robot) according to the control data, thereby controlling the cooking process. The output processing unit 102c may also output the control data to the drive unit 300 and the user. Furthermore, if the control data generation system 1 does not include a drive unit 300, the output processing unit 102c may output the control data to the user.

[0057] The storage unit 106 may store user data, recipe data, and the various measurement data described above. The storage unit 106 may also store information on how to control the cooking process according to the various measurement data described above. The storage unit 106 may also store information on the ideal taste, smell, or texture of a dish, categorized by dish, cooking method, and ingredients.

[0058] The cooking process information included in the recipe data may be detailed by a generating AI that has been given a prompt to further refine the cooking process information. The prompt may include an explanation of the concept of refinement and may also include a list of cooking utensils (corresponding to the cooking utensil data of the present invention) and a list of uses (corresponding to the use data of the present invention) as auxiliary information for refinement. See Figure 5 for examples of the list of cooking utensils and the list of uses.

[0059] The explanation of the concept of detailing may also be to make the generating AI understand the following (1) to (5). For the relationship between the four concepts of "recipe process," "task," "module," and "option," please refer to Figure 6, and for the relationship between the four concepts of "task," "module," "option," and "detailed process," please refer to Figure 7. For a specific example of a "module," please refer to Figure 8. (1) Breaking down each cooking process (recipe process) described in the recipe step by step into tasks and detailed processes. (2) A task is a process obtained by breaking down the cooking process described in the recipe one step at a time. (3) A detailed process is a process obtained by breaking down a task one step at a time, and is defined as a combination of modules and options. (4) A module is a cooking action on ingredients. (5) Options are information about the completion conditions of the cooking action, the cooking utensils to use, or any combination thereof, and the name of the ingredients.

[0060] The control data generation device 100 (or its control unit 102) may further include a recipe acquisition unit (not shown). The control data generation device 100 (or its control unit 102) may further include an information acquisition unit (not shown). The control data generation device 100 (or its control unit 102) may further include a prompt creation unit (not shown). The control data generation device 100 (or its control unit 102) may further include a recipe detailing unit (not shown). The control data generation device 100 (or its control unit 102) may further include a recipe detailing result evaluation unit (not shown). The control data generation device 100 (or its control unit 102) may further include an output processing unit (not shown).

[0061] The recipe acquisition unit acquires recipe data, including cooking process information, from external sources such as recipe websites. The information acquisition unit acquires a list of cooking utensils and a list of their uses from the storage unit 106 or an external storage device. The prompt creation unit creates prompts. When a prompt is created by the prompt creation unit, the recipe detailing unit provides the created prompt to the generating AI and acquires the detailed cooking process information output from the generating AI.

[0062] Furthermore, when the recipe detailing unit obtains the detailed cooking process information, the prompt creation unit may create a prompt that instructs the generating AI to further detail the cooking process information, and which includes an instruction to refer to the obtained detailing result. Alternatively, the control data generation device 100 (or its control unit 102) may repeatedly operate the prompt creation unit and the recipe detailing unit to further detail the cooking process information in the order of task level, module level, and detailed process level, while issuing instructions to refer to the obtained detailing results. For an example of the detailing process, please refer to Figures 9 and 10.

[0063] In an embodiment in which the control data generation device 100 (specifically the control unit 102) repeatedly operates the prompt creation unit and the recipe detailing unit to actually detail the recipe in the order of task level, module level, and detailed process level, while issuing instructions to refer to the acquired detailing results, the following are described at the end: [11. Prompt given to the generation AI during the first processing of recipe detailing], [12. Recipe detailing result output by the generation AI in the first processing of recipe detailing], [21. Prompt given to the generation AI during the second processing of recipe detailing], [22. Recipe detailing result output by the generation AI in the second processing of recipe detailing], [31. Prompt given to the generation AI during the third processing of recipe detailing], [32. Recipe detailing result output by the generation AI in the third processing of recipe detailing], [41. Prompt given to the generation AI during the fourth processing of recipe detailing], [42. Please refer to the following: [Recipe refinement results output by the generating AI in the fourth processing of recipe refinement], [51. Prompts given to the generating AI during the fifth processing of recipe refinement], and [52. Recipe refinement results output by the generating AI in the fifth processing of recipe refinement]. Note that in the last example, prompt creation and recipe refinement were repeated five times, but the number of repetitions is not limited to five and can be determined automatically or manually as needed. For example, this repetition process can be automatically continued or terminated by instructing the generating AI.

[0064] Furthermore, the refinement results obtained by the recipe refinement unit may be provided to the user or to the cooking robot. It is also possible to propose a refined cooking process tailored to the user's skill level. Moreover, by refining the recipe, it is possible to optimize and propose the level of detail of the recipe according to the user's skill level, not only for beginners but also for intermediate and advanced users.

[0065] The recipe refinement result evaluation unit evaluates the validity of the refinement results obtained by the recipe refinement unit. See Figure 10 for an example of the recipe refinement result evaluation process. The recipe refinement result evaluation unit may also evaluate whether the "task" and "detailed process" are appropriate. To evaluate appropriateness, "ground truth data (= detailed process recipes created by humans or judged to be appropriate by human eyes)" for representative recipes may be created, and the appropriateness of the recipe refinement results output by the generating AI may be evaluated by comparing them with the ground truth data (the comparison between the ground truth data and the recipe refinement results output by the generating AI may be performed by the generating AI). The entire recipe refinement and recipe refinement result evaluation process (see Figure 10) may be improved by feeding back insights into appropriate and inappropriate outputs to the prompt and RAG.

[0066] In particular, regarding the example of the recipe detailing result evaluation process, please refer to the following at the end: [61. Prompts given to the generating AI during the recipe detailing result evaluation process], [621. Evaluation results for the first task output by the generating AI in the recipe detailing result evaluation process], [622. Evaluation results for the second task output by the generating AI in the recipe detailing result evaluation process], [623. Evaluation results for the third task output by the generating AI in the recipe detailing result evaluation process], [624. Evaluation results for the fourth task output by the generating AI in the recipe detailing result evaluation process], [625. Evaluation results for the fifth task output by the generating AI in the recipe detailing result evaluation process], [631. Evaluation results for the first detailed step output by the generating AI in the recipe detailing result evaluation process], [632. Evaluation results for the second detailed step output by the generating AI in the recipe detailing result evaluation process], [633. Please refer to the evaluation results for the third detailed step output by the generating AI in the recipe detailing result evaluation process, [634. Evaluation results for the fourth detailed step output by the generating AI in the recipe detailing result evaluation process] and [635. Evaluation results for the fifth detailed step output by the generating AI in the recipe detailing result evaluation process]. In the last example, the output results were evaluated using an evaluation prompt to obtain evaluation results, and scores were obtained for each step from two perspectives: task and detailed step. In addition, in the last example, since the detailed tasks and detailed steps were evaluated for the recipes of the five recipe steps, evaluation results for 10 items (maximum 10 points) were obtained.

[0067] The output processing unit performs one of the following: outputting the refinement results obtained by the recipe refinement unit together with images of the cooking process; outputting the refinement results obtained by the recipe refinement unit together with a video of the cooking process; or both. For an overview of the processing performed by the output processing unit, please refer to the section titled "Image Generation of Cooking Procedures" in Figure 11.

[0068] As described above, according to this embodiment, it is possible to perform cooking while integrating and analyzing in real time the cooking conditions of the cooking robot, based on the real-time analysis of appearance measurement data, resistance measurement data, sound measurement data, olfactory measurement data, taste measurement data, and temperature measurement data measured in real time using various sensors during cooking. In other words, according to this embodiment, it is possible to integrate sensory information to predict the state of the cooked food (measured value conditions) and determine the end point of the cooking process based on the predicted value (the integration of sensory information may be achieved by statistical analysis or machine learning).

[0069] [3. Other Embodiments] Now, embodiments of the present invention have been described above, but the present invention may be implemented in various different embodiments within the scope of the technical idea described in the claims, in addition to the embodiments described above.

[0070] For example, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods.

[0071] In addition, the processing procedures, control procedures, specific names, information including parameters such as registration data and search conditions for each process, screen examples, and database configuration shown in the above-mentioned documents and drawings may be changed at will unless otherwise specified.

[0072] Furthermore, with respect to each device, the components shown in the illustrations are functional concepts and do not necessarily need to be physically configured as shown.

[0073] For example, the processing functions of the control data generation device 100, particularly those performed by the control unit 102, may be implemented in whole or in part by a CPU or GPU and a program interpreted and executed by the CPU or GPU, or they may be implemented as wired logic hardware. The program is recorded on a non-temporary computer-readable recording medium containing programmed instructions for causing the information processing device to execute the processing described in the embodiment, and is mechanically read by the control data generation device 100 as needed. That is, the storage unit 106, such as ROM or HDD, records a computer program that cooperates with the OS to give instructions to the CPU or GPU and perform various processing. This computer program is executed by being loaded into RAM and cooperates with the CPU or GPU to constitute the control unit.

[0074] Furthermore, this computer program may be stored on an application program server connected to the control data generation device 100 via any network, and it is possible to download all or part of it as needed.

[0075] Furthermore, the control data generation program for causing the information processing device to execute the control data generation method described in this embodiment, and the creation program for causing the information processing device to execute the creation method described in this embodiment (specifically, the method for creating a trained model and the method for creating a tuned model), may be stored in a computer-readable recording medium that is not temporary, or they may be configured as a program product. Here, "recording medium" includes any "portable physical medium" such as memory cards, USB memory, SD cards, flexible disks, magneto-optical disks, ROMs, EPROMs, EEPROMs, CD-ROMs, MOs, DVDs, and Blu-ray® Discs.

[0076] Furthermore, "program" refers to a data processing method described in any language or writing method, regardless of its format, such as source code or binary code. Note that "program" is not necessarily limited to a single, monolithic structure; it also includes distributed structures consisting of multiple modules or libraries, and those that work in cooperation with other programs, such as an operating system, to achieve their functions. Regarding the specific configuration and reading procedures for reading the recording medium in each device shown in the embodiments, as well as the installation procedures after reading, well-known configurations and procedures can be used.

[0077] The various databases stored in the storage unit 106 include memory devices such as RAM and ROM, fixed disk devices such as hard disks, flexible disks, and optical disks, and store various programs, tables, databases, and web page files used for various processes and website provision.

[0078] Furthermore, the control data generation device 100 may be configured as an information processing device such as a known personal computer or workstation, or as an information processing device to which any peripheral devices are connected. The control data generation device 100 may also be realized by implementing software (including programs or data, etc.) that implements the control data generation method described in this embodiment, or by further implementing software (including programs or data, etc.) that implements the creation method described in this embodiment (specifically, the method for creating a trained model and the method for creating a tuned model).

[0079] Furthermore, the specific forms of distribution and integration of the devices are not limited to those shown in the figures, and all or part of them can be configured by functionally or physically distributing and integrating them in any unit according to various additions or functional loads. In other words, the embodiments described above may be implemented in any combination, or the embodiments may be implemented selectively.

[0080] The present invention is particularly useful for real-time control of cooking processes in automatic cooking appliances or cooking robots.

[0081] 1 Control Data Generation System 100 Control Data Generation Device 102 Control Unit 102a Acquisition Unit 102b Generation Unit 102c Output Processing Unit 104 Communication Interface Unit 106 Storage Unit 108 Input / Output Interface Unit 112 Input Device 114 Output Device 200 Sensor 300 Drive Unit

[0082] [11. Prompt given to the generating AI during the first processing of recipe detailing] # Instructions You are a recipe expert. Detail the given recipe so that even a cooking novice can understand it. # Types of information provided - Recipe name: The name of the recipe - Ingredients: A list of ingredients and seasonings used in the recipe. (The amount of each ingredient is written in parentheses. The category of seasoning is written after the asterisk symbol.) - Cooking process: The steps to make the recipe (The level of detail and amount of information will vary depending on the menu) # Tips for recipe detailing Please use the categories included in the following four types of perspectives to detail the recipe. 1. Basic elements of a recipe: Elements obtained by dividing a recipe from the highest level of hierarchy. 2. Major elements: Elements included in the cooking process that is usually covered in most recipes: preparation, seasoning processing, heating, raw cooking, plating, and adding ingredients. 3. Minor elements: Elements included in the cooking process that is expected when the major elements are further broken down. 4. Modules: Elements obtained by dividing the actions that are common to cooking, regardless of the recipe, into the smallest units. # Output specifications Output only the components of the specified file in list format according to the instructions given by the user. Refer to the "Components to output" section for the components of the specified file. ## File Components - Row: The smallest unit of the recipe's cooking process - List of Columns - Column 1: No.: Number assigned to each cooking process - Column 2: Original Recipe: The process of the given recipe before it was broken down - Column 3: Task: The process obtained by breaking down the process of the given recipe - Column 4: Module: Category value of the module - Column 5: Cooking Utensils: Cooking utensils used in the detailed process - Column 6: Degree: Information such as heat level and water temperature in that cooking process - Column 7: Timing: Information indicating the cooking process in that cooking process - Column 8: Ingredient State [Before Cooking Process]: The state of the ingredients before performing that cooking process. (To inform the cook of the state before and after that process) - Column 9: Ingredient State [After Cooking Process]: The state of the ingredients after performing that cooking process. (To inform the cook of the state before and after that process) - Column 10: Detailed Process: A detailed description of the cooking process, which is the smallest unit obtained by dividing the given recipe.This document uses the information from columns 2 through 7 to explain the smallest unit of the cooking process in a way that is easy for even beginner cooks to understand. # Components to output: Break down the steps of the original recipe down to the task level. Separate each task into a new row. A task refers to a level of detail that includes multiple actions, such as "wash the rice" or "cut the vegetables into ginkgo leaf shapes." ## Components of the file: Separate the columns with a colon ":" and output data targeting only the specified columns below. The first row should output the column names. - Row: The smallest unit of the cooking process in the recipe - List of columns - Column 1: No.: Assign a number to each cooking process - Column 2: Original recipe: The process of the given recipe before refinement (If the output is the same as the previous column, output "↓" (downward arrow)) - Column 3: Task: The process of the given recipe broken down # Recipe name - Authentic Chicken Curry # Ingredients - Chicken thigh meat with bone [1500g] - Salt [to taste] - Pepper [to taste] - Salad oil [to taste] - Onion [1] - Carrot [1] - Celery [1] - Chicken carcass, shredded [2] *Ingredient group A - Water [3 liters] *Ingredient group A - Whole black peppercorns

[10] *Ingredient group B - Bouquet garni [1] *Ingredient group B - Beef consommé [2] *Ingredient group B - Coconut flakes [100g] *Ingredient Group B - Apple [150g] *Ingredient Group C - Onion [250g] *Ingredient Group C - Ginger [50g] *Ingredient Group C - Garlic [40g] *Ingredient Group C - Mango Chutney [100g] *Ingredient Group C - Curry Powder [80g] *Ingredient Group C - Red Chili Peppers [2] *Ingredient Group C - Cinnamon Stick [1] *Ingredient Group C - Curry Powder [1 tbsp] *Seasoning D - Umami Broth [to taste] *Ingredient Group D - Salt [a little] *Ingredient Group D - Pepper [a little] *Ingredient Group D - Carrot [to taste] *Ingredient Group E - Broccoli [to taste] *Ingredient Group E - Potato [to taste] *Ingredient Group E - Rice [440g] - Sliced ​​Almonds [to taste] # Cooking Process 1. Slice the onion, carrot, and celery thinly.2. Put the onion, carrot, and celery from (1) and A into a pot and heat it. Once it boils, remove the scum. Reduce the heat, add B, and simmer until the soup is reduced to about 2 / 3 of its original volume. Strain the soup through a sieve and return it to the pot. 3. Peel and core the apple from C, and cut it into appropriate sizes along with the onion. Blend it with the other C ingredients in a blender until it becomes a paste, and add it to the soup from (2). 4. Season the chicken with salt and pepper. Heat oil in a frying pan and brown the surface of the chicken. Add it to (3), cover, and simmer over low heat for 30-40 minutes. 5. Add D to (4) and adjust the taste. Cut the vegetables from E into bite-sized pieces and steam them in a steamer. Serve rice in a bowl, sprinkle with roasted almonds, add the vegetables, and pour the curry from (4) over it.

[0083] [12. Recipe refinement results output by the generating AI in the first processing of recipe refinement] No.: Original recipe: Task 1: Slice the onion, carrot and celery thinly. : Slice the onion thinly. 2:↓: Slice the carrot thinly. 3:↓: Slice the celery thinly. 4: Put the onion, carrot and celery from (1) and A into a pot and heat it, and remove the scum when it boils. Reduce the heat, add B and simmer until the amount of soup is reduced to about 2 / 3. Strain it through a sieve and return it to the pot. : Put the thinly sliced ​​onion, carrot and celery into the pot. 5:↓: Put the shredded chicken bones and water into the pot. 6:↓: Heat the pot and bring it to a boil. 7:↓: Remove the scum when it boils. 8: Reduce the heat and add the black peppercorns, bouquet garni, Knorr Beef Consommé, and coconut flakes. 9: Simmer until the soup is reduced to 2 / 3 of its original volume. 10: Strain the soup through a sieve. 11: Return the soup to the pot. 12: Peel and core the apples from C, and cut them into appropriate sizes along with the onions. Blend them together with the other C ingredients in a blender until smooth, and add to the soup from (2). : Peel the apples. 13: Remove the cores from the apples. 14: Cut the apples into appropriate sizes. 15: Cut the onions into appropriate sizes. 16: Blend the apples, onions, ginger, garlic, mango chutney, curry powder, red chili peppers, and cinnamon sticks in a blender until smooth, and add to the soup. 18: Season the chicken with salt and pepper. Heat oil in a frying pan and brown the surface of the chicken. Add to (3), cover and simmer over low heat for 30-40 minutes. : Season the chicken with salt and pepper. 19:↓: Heat salad oil in a frying pan. 20:↓: Brown the surface of the chicken in the frying pan. 21:↓: Add the browned chicken to the soup in (3). 22:↓: Cover the pot and simmer over low heat for 30-40 minutes. 23: Add D to (4) and adjust the taste. Cut the vegetables E into bite-sized pieces and steam them in a steamer. Serve rice in a bowl, sprinkle with roasted almonds, add the vegetables, and pour the curry from (4) over it. : Add "umami stock", salt and pepper to the pot from (4) and adjust the taste. 24: Cut the carrots, broccoli, and potatoes into bite-sized pieces. 25: Steam the cut vegetables in a steamer.26: Put rice into a bowl. 27: Sprinkle roasted almonds over the rice. 28: Add vegetables to the bowl. 29: Pour the curry from (4) over the bowl.

[0084] [21. Prompt given to the generating AI during the second processing of recipe detailing] # Components to be output Add to the previous output result and output the modules included for each task. A module refers to a level of granularity that breaks down the task "washing rice" to individual actions of the cook, such as "prepare a bowl" and "put rice in the bowl". Create the modules while referring to the tabular information provided in the "# Reference information for vegetable preparation" section. ## Output steps 1. Read the output instructions below and understand the module concept. 2. Read the "# Reference information for vegetable preparation" section and understand the preparation of the vegetables that appear in the recipe. 3. Output the module with an understanding of the module concept and vegetable preparation. 4. Check whether the output module is unnatural from a cooking procedure standpoint and correct it as necessary. ## Module output format Modules should be able to use common notation for any recipe, so do not include recipe-specific expressions or descriptions of ingredients. Output the actions within the module using generalized expressions. ## Output data from only the specified columns, separated by colons ":" between the constituent columns of the file. The first row should output the column names.- Row: The smallest unit of the cooking process in the recipe - List of columns - Column 1: No.: Assign a number to each cooking process - Column 2: Original recipe: The process of the given recipe before refinement (If the output is the same as the previous column, output "↓" (downward arrow)) - Column 3: Task: The process obtained by breaking down the process of the given recipe (If the output is the same as the previous column, output "↓" (downward arrow)) - Column 4: Module: Category value of the module # Recipe name - Authentic Chicken Curry # Ingredients - Chicken thigh meat with bone [1500g] - Salt [appropriate amount] - Pepper [appropriate amount] - Salad oil [appropriate amount] - Onion [1] - Carrot [1] - Celery [1] - Chicken carcass, shredded [2] *Ingredient group A - Water [3 liters] *Ingredient group A - Whole black peppercorns

[10] *Ingredient group B - Bouquet garni [1 piece] *Ingredient Group B - Beef Consommé [2 pieces] *Ingredient Group B - Coconut Flakes [100g] *Ingredient Group B - Apple [150g] *Ingredient Group C - Onion [250g] *Ingredient Group C - Ginger [50g] *Ingredient Group C - Garlic [40g] *Ingredient Group C - Mango Chutney [100g] *Ingredient Group C - Curry Powder [80g] *Ingredient Group C - Red Chili Peppers [2] *Ingredient Group C - Cinnamon Stick [1] *Ingredient Group C - Curry Powder [1 tablespoon] *Seasoning D - Umami Broth [to taste] *Ingredient Group D - Salt [a little] *Ingredient Group D - Pepper [a little] *Ingredient Group D - Carrot [to taste] *Ingredient Group E - Broccoli [to taste] *Ingredient Group E - Potato [to taste] *Ingredients Group E - Rice [440g] - Sliced ​​almonds [appropriate amount] # Cooking process 1. Slice the onion, carrot and celery thinly. 2. Put the sliced ​​onion, carrot and celery from (1) and A into a pot and heat it. Once it boils, remove the scum. Reduce the heat, add B, and simmer until the soup is reduced to about 2 / 3 of its original volume. Strain it through a sieve and return it to the pot. 3. Peel and core the apple from C, and cut it into appropriate sizes along with the onion. Blend it with the other C ingredients in a blender until it becomes a paste, and add it to the soup from (2). 4. Season the chicken with salt and pepper.Heat oil in a frying pan and brown the surface of the chicken. Add to (3), cover and simmer over low heat for 30-40 minutes. 5. Add D to (4) and adjust the seasoning. Cut the vegetables E into bite-sized pieces and steam them in a steamer. Serve rice in a bowl, sprinkle with roasted almonds, add the vegetables, and pour the curry from (4) over it. # Reference information for preparing vegetables The table below lists the cooking utensils used to prepare the vegetables used in this recipe. Items are separated by the symbol "|", and if a cooking utensil is used in a combination of vegetables and preparation steps, it is indicated with "○". Vegetables | Process | Hands | Knife | Cutting board | Kitchen scissors | Peeler | Chopsticks | Spatula | Spoon | Bowl | Colander | Pot | Stove | Sliced ​​onion | Peel | ○ | | | | | | | | | | | | Onion | Remove the stem | | ○ | ○ | | | | | | | | | |.

[0085] [22. Recipe refinement results output by the generating AI in the second processing of recipe refinement] No.: Original recipe: Task: Module 1: Slice the onion, carrot and celery thinly. : Slice the onion thinly. : Wash the ingredients 2: ↓: ↓: Peel the skin 3: ↓: ↓: Remove the stem 4: ↓: ↓: Prepare the knife 5: ↓: ↓: Prepare the cutting board 6: ↓: ↓: Cut the ingredients 7: ↓: Slice the carrot thinly. : Wash the ingredients 8: ↓: ↓: Peel the skin 9: ↓: ↓: Remove the stem 10: ↓: ↓: Prepare the knife 11: ↓: ↓: Prepare the cutting board 12: ↓: ↓: Cut the ingredients 13: ↓: Slice the celery thinly. : Wash the ingredients 14:↓:↓: Separate the leaves and stems 15:↓:↓: Remove the strings 16:↓:↓: Prepare the knife 17:↓:↓: Prepare the cutting board 18:↓:↓: Cut the ingredients 19: Put the onion, carrot, and celery from (1) and A into a pot and heat it, and remove the scum when it boils. : Prepare the pot 20:↓:↓: Add the ingredients 21:↓:↓: Add water 22:↓:↓: Prepare the stove 23:↓:↓: Heat the pot 24:↓:↓: Remove the scum 25: Reduce the heat, add B, and simmer until the amount of soup is reduced to about 2 / 3. : Reduce the heat 26:↓:↓: Add the ingredients 27:↓:↓: Simmer the soup 28: Strain it through a sieve and return it to the pot. : Prepare a strainer 29:↓:↓: Strain the soup 30:↓:↓: Put it back into the pot 31: Peel the apples from C, remove the cores, and cut them into appropriate sizes along with the onions. : Wash the apples 32:↓:↓: Peel the apples 33:↓:↓: Remove the cores 34:↓:↓: Prepare a knife 35:↓:↓: Prepare a cutting board 36:↓:↓: Cut the ingredients 37:↓:↓: Cut the onions into appropriate sizes 38: Blend the onions together with the other C ingredients in a blender to make a paste, and add it to the soup from (2). : Prepare a blender 39:↓:↓: Put the ingredients in 40:↓:↓: Blend 41:↓:↓: Add the paste to the soup 42: Season the chicken with salt and pepper. 43: Season the chicken with salt and pepper. Heat oil in a frying pan and brown the surface of the chicken. Prepare the frying pan. Add oil. Heat the frying pan. Add the chicken. Cook the chicken. Add the chicken to (3), cover, and simmer over low heat for 30-40 minutes.: Add chicken to soup 49: ↓:↓: Cover 50: ↓:↓: Simmer over low heat 51: Add D to (4) and adjust the taste. : Add seasonings to the pot 52: ↓:↓: Adjust the taste 53: Cut the vegetables E into bite-sized pieces and steam them in a steamer. : Wash the carrots 54: ↓:↓: Wash the broccoli 55: ↓:↓: Wash the potatoes 56: ↓:↓: Cut the carrots into bite-sized pieces 57: ↓:↓: Cut the broccoli into bite-sized pieces 58: ↓:↓: Peel the potatoes 59: ↓:↓: Cut the potatoes into bite-sized pieces 60: ↓:↓: Prepare the steamer 61: ↓:↓: Steam the vegetables 62: Put rice in a bowl, sprinkle with roasted almonds, add the vegetables, and pour the curry from (4) over it. 63: Prepare a bowl ↓:↓: Add rice 64: ↓:↓: Sprinkle with almonds 65: ↓:↓: Add vegetables 66: ↓:↓: Pour curry over it.

[0086] [31. Prompt given to the generating AI during the third processing of recipe detailing] # Components to be output Add the names of cooking utensils used in each module to the previous output result. Consider the cooking utensils while referring to the tabular information provided in the "# Reference information for vegetable preparation" and "# Reference information for cooking utensils" sections. ## Output steps 1. Read the output instructions below and understand the concept of cooking utensils. 2. Read the "# Reference information for vegetable preparation" section and understand the combination of vegetable preparation and cooking utensils that appear in the recipe. 3. Output the cooking utensils with an understanding of the concept of cooking utensils and vegetable preparation. 4. Check that there are no omissions or errors in the outputted cooking utensils and correct them as necessary. ## Types of cooking utensils Output all cooking utensils used in each module. (The output should be the specific names of the cooking utensils.) ## Components of the file Output data targeting only the following specified columns, separated by colon symbols ":". The first row should output the column names.- Row: The smallest unit of cooking process in the recipe - List of columns - Column 1: No.: Number each cooking process - Column 2: Original recipe: Process of the given recipe before refinement - Column 3: Task: Processes obtained by breaking down the process of the given recipe - Column 4: Module: Category value of the module - Column 5: Cooking utensils: Cooking utensils used in the detailed process # Recipe name - Authentic Chicken Curry # Ingredients - Chicken thigh meat with bone [1500g] - Salt [to taste] - Pepper [to taste] - Salad oil [to taste] - Onion [1] - Carrot [1] - Celery [1] - Chicken carcass, shredded [2] *Ingredient group A - Water [3 liters] *Ingredient group A - Whole black peppercorns

[10] *Ingredient group B - Bouquet garni [1] *Ingredient group B - Beef consommé [2] *Ingredient group B - Coconut flakes [100g] *Ingredient Group B - Apple [150g] *Ingredient Group C - Onion [250g] *Ingredient Group C - Ginger [50g] *Ingredient Group C - Garlic [40g] *Ingredient Group C - Mango Chutney [100g] *Ingredient Group C - Curry Powder [80g] *Ingredient Group C - Red Chili Peppers [2] *Ingredient Group C - Cinnamon Stick [1] *Ingredient Group C - Curry Powder [1 tbsp] *Seasoning D - Umami Broth [to taste] *Ingredient Group D - Salt [a little] *Ingredient Group D - Pepper [a little] *Ingredient Group D - Carrot [to taste] *Ingredient Group E - Broccoli [to taste] *Ingredient Group E - Potato [to taste] *Ingredient Group E - Rice [440g] - Sliced ​​Almonds [to taste] # Cooking Process 1. 1. Slice the onion, carrot, and celery thinly. 2. Put the sliced ​​onion, carrot, and celery from (1) and ingredients A into a pot and bring to a boil, then skim off any foam. Reduce the heat, add ingredients B, and simmer until the soup is reduced to about two-thirds of its original volume. Strain the soup through a sieve and return it to the pot. 3. Peel and core the apple from ingredient C, and cut it into appropriate sizes along with the onion. Blend the apple and the other ingredients C in a blender until smooth and add to the soup from (2). 4. Season the chicken with salt and pepper. Heat oil in a frying pan and brown the surface of the chicken. Add the chicken to (3), cover, and simmer over low heat for 30-40 minutes.5. Add D to (4) and adjust the taste. Cut the vegetables E into bite-sized pieces and steam them in a steamer. Serve the rice in a bowl, sprinkle with roasted almonds, add the vegetables, and pour the curry from (4) over it. # Reference information for preparing vegetables The table below lists the cooking utensils used to prepare the vegetables used in this recipe. Items are separated by the symbol "|", and if a cooking utensil is used in a combination of vegetables and preparation steps, it is indicated with "○". Vegetables | Step | Hands | Knife | Cutting board | Kitchen scissors | Peeler | Chopsticks | Spatula | Spoon | Bowl | Colander | Pot | Stove | Sliced ​​onion | Peel | ○ | | | | | | | | | | | | Onion | Remove stem | | ○ | ○ | | | | | | | | | | # Reference information for cooking utensils The table below lists the cooking utensils used in this recipe and their uses. The items are separated by the "|" symbol, and each item describes a cooking utensil and its intended use. Cooking Utensil Name | Use Knife | Used for cutting ingredients Cutting board | Used as a base when cutting ingredients Peeler | Used for peeling vegetables and fruits.

[0087] [32. Recipe detailing results output by the generating AI in the third processing of recipe detailing] No.: Original recipe: Task: Module: Cooking utensils 1: Slice the onion, carrot and celery thinly. : Slice the onion thinly. : Wash the ingredients: Hands 2: ↓: ↓: Peel: Hands 3: ↓: ↓: Remove the stem: Knife, cutting board 4: ↓: ↓: Prepare the knife: Knife 5: ↓: ↓: Prepare the cutting board: Cutting board 6: ↓: ↓: Cut the ingredients: Knife, cutting board 7: ↓: Slice the carrot thinly. : Wash the ingredients: by hand 8: Peel the skin: peeler 9: Remove the stems: knife, cutting board 10: Prepare the knife: knife 11: Prepare the cutting board: cutting board 12: Cut the ingredients: knife, cutting board 13: Slice the celery thinly. : Wash the ingredients: by hand 14: Separate the leaves and stems: knife, cutting board 15: Remove the strings: knife, cutting board 16: Prepare the knife: knife 17: Prepare the cutting board: cutting board 18: Cut the ingredients: knife, cutting board 19: Put the onions, carrots, and celery from (1) and A into a pot and heat it, then remove the scum when it boils. :Prepare a pot: pot 20:↓:↓:Add ingredients: pot 21:↓:↓:Add water: measuring cup 22:↓:↓:Prepare a stove: stove 23:↓:↓:Heat the pot: stove 24:↓:↓:Remove the scum: skim off the scum 25:Reduce the heat, add B, and simmer until the soup is reduced to about 2 / 3 of its original volume. :Reduce the heat: stove 26:↓:↓:Add ingredients: hand 27:↓:↓:Simmer the soup: pot, stove 28:Strain through a sieve and return to the pot. :Prepare a sieve: sieve 29:↓:↓:Strain the soup: sieve 30:↓:↓:Return to the pot: pot 31:Peel the apples from C, remove the cores, and cut them into appropriate sizes along with the onions. : Wash the apple: by hand 32: Peel it: knife, cutting board 33: Remove the core: knife, cutting board 34: Prepare the knife: knife 35: Prepare the cutting board: cutting board 36: Cut the ingredients: knife, cutting board 37: Cut the onion into appropriate sizes: knife, cutting board 38: Blend with the other C ingredients in a blender until it becomes a paste, and add to the soup from (2).: Prepare a blender: Blender 39: ↓: ↓: Add ingredients: Blender 40: ↓: ↓: Blend: Blender 41: ↓: ↓: Add paste to soup: Hand 42: Season chicken with salt and pepper. : Season chicken with salt and pepper: Hand 43: Heat oil in a frying pan and brown the surface of the chicken. : Prepare a frying pan: Frying pan 44: ↓: ↓: Add oil: Measuring spoon 45: ↓: ↓: Heat the frying pan: Stove 46: ↓: ↓: Add chicken: Tongs 47: ↓: ↓: Brown chicken: Frying pan 48: Add to (3), cover and simmer over low heat for 30-40 minutes. : Add chicken to soup: pot 49: ↓: ↓: Cover with a lid: pot lid 50: ↓: ↓: Simmer over low heat: stove 51: Add D to (4) and adjust the taste. : Add seasonings to the pot: measuring spoon 52: ↓: ↓: Adjust the taste: chopsticks 53: Cut the vegetables E into bite-sized pieces and steam in a steamer. : Wash the carrots: by hand 54:↓:↓: Wash the broccoli: by hand 55:↓:↓: Wash the potatoes: by hand 56:↓:↓: Cut the carrots into bite-sized pieces: knife, cutting board 57:↓:↓: Cut the broccoli into bite-sized pieces: knife, cutting board 58:↓:↓: Peel the potatoes: peeler 59:↓:↓: Cut the potatoes into bite-sized pieces: knife, cutting board 60:↓:↓: Prepare the steamer: steamer 61:↓:↓: Steam the vegetables: steamer 62: Put rice in a bowl, sprinkle with roasted almonds, add the vegetables, and pour the curry from (4) over it. : Prepare a bowl: Tray 63: ↓: ↓: Serve the rice: Rice paddle 64: ↓: ↓: Sprinkle almonds: Hand 65: ↓: ↓: Add vegetables: Chopsticks 66: ↓: ↓: Pour the curry over it: Ladle.

[0088] [41. Prompt given to the generating AI during the fourth processing of recipe detailing] # Output the "detailed steps" of the recipe based on the source recipe and the already outputted task, cooking equipment, and module information of the components to be output. ## Output data targeting only the following specified columns, separated by colon symbols ":" between component columns. Output the column names in the first row. - Row: The smallest unit of the recipe's cooking steps - List of columns - Column 1: No.: Assign a number to each cooking step - Column 2: Source recipe: Steps of the given recipe before detailing - Column 3: Task: Steps obtained by breaking down the steps of the given recipe - Column 4: Module: Category value of the module - Column 5: Cooking equipment: Cooking equipment used in the detailed steps - Column 10: Detailed steps: A detailed explanation of the cooking steps, which are obtained by dividing the given recipe down to the smallest unit. Use the information from columns 2 to 7 to create a text that explains the smallest unit of the cooking steps in a way that is easy for even cooking beginners to understand.# Recipe Name - Authentic Chicken Curry # Ingredients - Chicken bone-in thigh meat cut into pieces [1500g] - Salt [to taste] - Pepper [to taste] - Salad oil [to taste] - Onion [1] - Carrot [1] - Celery [1] - Shredded chicken carcass [2] *Ingredient Group A - Water [3 liters] *Ingredient Group A - Whole black peppercorns

[10] *Ingredient Group B - Bouquet garni [1] *Ingredient Group B - Beef consommé [2] *Ingredient Group B - Coconut flakes [100g] *Ingredient Group B - Apple [150g] *Ingredient Group C - Onion [250g] *Ingredient Group C - Ginger [50g] *Ingredient Group C - Garlic [40g] *Ingredient Group C - Mango chutney [100g] *Ingredient Group C - Curry powder [80g] *Ingredient group C - Red chili peppers [2] *Ingredient group C - Cinnamon stick [1] *Ingredient group C - Curry powder [1 tablespoon] *Seasoning D - Umami stock [appropriate amount] *Ingredient group D - Salt [a little] *Ingredient group D - Pepper [a little] *Ingredient group D - Carrots [appropriate amount] *Ingredient group E - Broccoli [appropriate amount] *Ingredient group E - Potatoes [appropriate amount] *Ingredient group E - Rice [440g] - Sliced ​​almonds [appropriate amount] # Cooking process 1. Slice the onion, carrots, and celery thinly. 2. Put the sliced ​​onion, carrots, and celery from (1) and ingredients A into a pot and heat it. Once it boils, remove the scum. Reduce the heat, add ingredients B, and simmer until the amount of soup is reduced to about 2 / 3. Strain it through a sieve and return it to the pot. 3. Peel and core the apple from C, and cut it into bite-sized pieces along with the onion. Blend them together with the other C ingredients in a blender until smooth, then add to the soup from (2). 4. Season the chicken with salt and pepper. Heat oil in a frying pan and brown the surface of the chicken. Add to (3), cover, and simmer over low heat for 30-40 minutes. 5. Add D to (4) and adjust the seasoning. Cut the vegetables from E into bite-sized pieces and steam them in a steamer. Serve the rice in a bowl, sprinkle with roasted almonds, add the vegetables, and pour the curry from (4) over it.

[0089] [42. Recipe detailing results output by the generating AI in the fourth processing of recipe detailing] No.: Original recipe: Task: Module: Cooking utensils: Detailed steps 1: Slice the onion, carrot and celery thinly. : Slice the onion thinly. : Wash the ingredients: Hands: Wash the onion under running water to remove dirt. 2: ↓: ↓: Peel: Hands: Peel the onion by hand. 3: ↓: ↓: Remove the stem: Knife, cutting board: Use a knife to remove the stem of the onion on the cutting board. 4: ↓: ↓: Prepare the knife: Knife: Prepare the knife. 5: ↓: ↓: Prepare the cutting board: Cutting board: Prepare the cutting board. 6: ↓: ↓: Cut the ingredients: Knife, cutting board: Slice the onion thinly. 7: ↓: Slice the carrot thinly. 1. Wash the ingredients: Hands: Wash the carrots under running water to remove dirt. 8. Peel the skin: Peeler: Use a peeler to peel the carrots. 9. Remove the top: Knife, cutting board: Use a knife to remove the top of the carrots on the cutting board. 10. Prepare the knife: Knife: Prepare the knife. 11. Prepare the cutting board: Cutting board: Prepare the cutting board. 12. Cut the ingredients: Knife, cutting board: Slice the carrots thinly. 13. Slice the celery thinly. Wash the ingredients: Hands: Wash the celery under running water to remove dirt. 14. Separate the leaves and stems: Knife, cutting board: Use a knife to separate the celery leaves and stems on the cutting board. 15:↓:↓:Remove strings:Knife, cutting board:Use a knife to remove the strings from the celery. 16:↓:↓:Prepare a knife:Knife:Prepare a knife. 17:↓:↓:Prepare a cutting board:Cutting board:Prepare a cutting board. 18:↓:↓:Cut the ingredients:Knife, cutting board:Slice the celery thinly. 19:Put the onion, carrot, and celery from (1) and A into a pot and heat it, then remove the scum once it boils.:Prepare a pot:Pot:Prepare a pot. 20:↓:↓:Add ingredients:Pot:Put the thinly sliced ​​onion, carrot, and celery into the pot. 21:↓:↓:Add water:Measuring cup:Measure the water with a measuring cup and pour it into the pot. 22:↓:↓:Prepare a stove:Stove:Prepare a stove. 23:↓:↓: Heat the pot: Stove: Place the pot on the stove and heat it. 24:↓:↓: Remove the scum: Skim off the scum: Use a skimmer to remove the scum that rises to the surface when the pot boils.25: Reduce the heat, add B, and simmer until the soup is reduced to about 2 / 3 of its original volume. : Reduce the heat: Stove: Reduce the heat on the pot. 26: ↓: ↓: Add the ingredients: Hand: Add the black peppercorns, bouquet garni, beef consommé, and coconut flakes to the pot. 27: ↓: ↓: Reduce the soup: Pot, stove: Simmer until the soup is reduced to 2 / 3 of its original volume. 28: Strain through a sieve and return to the pot. : Prepare a sieve: Sieve: Prepare a sieve. 29: ↓: ↓: Strain the soup: Sieve: Strain the soup through a sieve. 30: ↓: ↓: Return to the pot: Pot: Return the strained soup to the pot again. 31: Peel and core the apples from C, and cut them into appropriate sizes along with the onions. : Wash the apple: Hands: Wash the apple under running water to remove any dirt. 32: Peel: Knife, cutting board: Use a knife to peel the apple. 33: Remove the core: Knife, cutting board: Use a knife to remove the core from the apple on the cutting board. 34: Prepare the knife: Knife: Prepare the knife. 35: Prepare the cutting board: Cutting board: Prepare the cutting board. 36: Cut the ingredients: Knife, cutting board: Cut the apple into appropriate sizes. 37: Cut the onion into appropriate sizes: Knife, cutting board: Cut the onion into appropriate sizes. 38: Blend with the other C ingredients in a blender until it becomes a paste, then add to the soup from (2). : Prepare the blender: Blender: Prepare the blender. 39: ↓:↓: Add ingredients: Blender: Put apple, onion, ginger, garlic, mango chutney, curry powder, red chili pepper, and cinnamon stick into a blender. 40: ↓:↓: Blend: Blender: Blend the ingredients into a paste. 41: ↓:↓: Add paste to soup: Hand: Add the paste to the soup. 42: Season chicken with salt and pepper. : Season chicken with salt and pepper: Hand: Season the chicken with salt and pepper. 43: Heat oil in a frying pan and brown the surface of the chicken. : Prepare frying pan: Frying pan: Prepare a frying pan. 44: ↓:↓: Add oil: Measuring spoon: Measure the salad oil with a measuring spoon and put it into the frying pan. 45: ↓:↓: Heat frying pan: Stove: Heat the frying pan on the stove.46: Add the chicken: tongs: Use tongs to put the chicken into the frying pan. 47: Brown the chicken: frying pan: brown the surface of the chicken. 48: Add to (3), cover and simmer over low heat for 30-40 minutes.: Add the chicken to the soup: pot: Add the browned chicken to the soup. 49: Cover: pot lid: Cover the pot. 50: Simmer over low heat: stove: Simmer over low heat for 30-40 minutes. 51: Add D to (4) and adjust the taste.: Add the seasonings to the pot: measuring spoon: Measure the umami stock, salt and pepper with measuring spoons and add them to the pot. 52: Adjust the taste: chopsticks: Stir the soup with chopsticks and adjust the taste. 53: Cut the vegetables from E into bite-sized pieces and steam them in a steamer. : Wash the carrots: by hand: Wash the carrots under running water to remove dirt. 54: ↓: ↓: Wash the broccoli: by hand: Wash the broccoli under running water to remove dirt. 55: ↓: ↓: Wash the potatoes: by hand: Wash the potatoes under running water to remove dirt. 56: ↓: ↓: Cut the carrots into bite-sized pieces: knife, cutting board: Cut the carrots into bite-sized pieces. 57: ↓: ↓: Cut the broccoli into bite-sized pieces: knife, cutting board: Cut the broccoli into bite-sized pieces. 58: ↓: ↓: Peel the potatoes: peeler: Use a peeler to peel the potatoes. 59: ↓: ↓: Cut the potatoes into bite-sized pieces: knife, cutting board: Cut the potatoes into bite-sized pieces. 60: ↓:↓: Prepare the steamer: Steamer: Prepare the steamer. 61: ↓:↓: Steam the vegetables: Steamer: Steam the vegetables in the steamer. 62: Put rice in a bowl, sprinkle with roasted almonds, add the vegetables, and pour the curry from (4) over it. : Prepare a bowl: Tray: Prepare a bowl for serving. 63: ↓:↓: Serve the rice: Rice paddle: Use a rice paddle to put the rice into the bowl.

[0090] [51. Prompt given to the generating AI during the 5th processing of recipe detailing] # Output the degree, timing, and ingredient state [before cooking process] and ingredient state [after cooking process] related to each "detailed process" component you want to output. Do not change the content of the "detailed process". Check again whether the degree, timing, and ingredient state [before cooking process] and ingredient state [after cooking process] that you have output once are consistent with the "detailed process", and make corrections as needed. ## Output data targeting only the following specified columns, separated by colon symbols ":" between component columns. Output the column names in the first row. - Row: The smallest unit of the cooking process in the recipe - List of columns - Column 1: No.: Assign a number to each cooking process - Column 2: Original recipe: The steps of the given recipe before it was broken down - Column 3: Task: The steps obtained by breaking down the steps of the given recipe - Column 4: Module: Category value of the module - Column 5: Cooking utensils: Cooking utensils used in the detailed process - Column 6: Degree: Information such as heat level and water temperature in that cooking process - Column 7: Timing: Information on the cooking guidelines in that cooking process - Column 8: Ingredient state [Before cooking process]: The state of the ingredients before performing that cooking process. (To inform the cook of the state before and after that process) - Column 9: Ingredient state [After cooking process]: The state of the ingredients after performing that cooking process. (To inform the cook of the state before and after that process) - Column 10: Detailed process: A detailed explanation of the cooking process, which is the smallest unit of the given recipe. Using the information from columns 2 to 7, this is a text that explains the smallest unit of the cooking process in a way that is easy for even cooking beginners to understand. ### "Ingredient State [Before Cooking]" and "Ingredient State [After Cooking]" The ingredient state consists of six elements composed of two axes: "Ingredient" and "State". Express these elements as a single concise sentence and output "Ingredient State [Before Cooking]" and "Ingredient State [After Cooking]".- Ingredients: Name, Category, Part - Condition: Shape, Processing, Quantity # Recipe Name - Authentic Chicken Curry # Ingredients - Chicken bone-in thigh meat cut into pieces [1500g] - Salt [to taste] - Pepper [to taste] - Salad oil [to taste] - Onion [1] - Carrot [1] - Celery [1] - Shredded chicken carcass [2] *Ingredient Group A - Water [3 liters] *Ingredient Group A - Whole black peppercorns

[10] *Ingredient Group B - Bouquet garni [1] *Ingredient Group B - Beef consommé [2] *Ingredient Group B - Coconut flakes [100g] *Ingredient Group B - Apple [150g] *Ingredient Group C - Onion [250g] *Ingredient Group C - Ginger [50g] *Ingredient Group C - Garlic [40g] *Ingredient Group C - Mango chutney [100g] *Ingredient group C - Curry powder [80g] *Ingredient group C - Red chili peppers [2] *Ingredient group C - Cinnamon stick [1] *Ingredient group C - Curry powder [1 tablespoon] *Seasoning D - Umami stock [appropriate amount] *Ingredient group D - Salt [a little] *Ingredient group D - Pepper [a little] *Ingredient group D - Carrots [appropriate amount] *Ingredient group E - Broccoli [appropriate amount] *Ingredient group E - Potatoes [appropriate amount] *Ingredient group E - Rice [440g] - Sliced ​​almonds [appropriate amount] # Cooking process 1. Slice the onion, carrots, and celery thinly. 2. Put the sliced ​​onion, carrots, and celery from (1) and ingredients A into a pot and heat it. Once it boils, remove the scum. Reduce the heat, add B, and simmer until the soup is reduced to about 2 / 3 of its original volume. Strain through a sieve and return to the pot. 3. Peel and core the apples from C, and cut them into appropriate sizes along with the onions. Blend them together with the other C ingredients in a blender until smooth, and add to the soup from (2). 4. Season the chicken with salt and pepper. Heat oil in a frying pan and brown the surface of the chicken. Add to (3), cover, and simmer over low heat for 30-40 minutes. 5. Add D to (4) and adjust the seasoning. Cut the vegetables from E into bite-sized pieces and steam them in a steamer. Serve rice in a bowl, sprinkle with roasted almonds, add the vegetables, and pour the curry from (4) over it.

[0091] [52. Recipe detailing results output by the generating AI in the 5th processing of recipe detailing] No.: Original recipe: Task: Module: Cooking utensils: Degree: Timing: Ingredient state [Before cooking process]: Ingredient state [After cooking process]: Detailed process 1: Slice the onion, carrot and celery thinly. : Slice the onion thinly. : Wash the ingredients: Hands::: Onion [Unwashed]: Onion [Washed]: Wash the onion under running water to remove dirt. 2: ↓: ↓: Peel: Hands::: Onion [Washed]: Onion [No skin]: Peel the onion by hand. 3: ↓: ↓: Remove the stem: Knife, cutting board::: Onion [No skin]: Onion [No stem]: Use a knife to remove the stem of the onion on the cutting board. 4: ↓:↓: Prepare the knife: Knife::::: Prepare the knife. 5: ↓:↓: Prepare the cutting board: Cutting board::::: Prepare the cutting board. 6: ↓:↓: Cut the ingredients: Knife, cutting board::: Onion [without stem]: Onion [thinly sliced]: Slice the onion thinly. 7: ↓: Slice the carrot thinly. : Wash the ingredients: Hands: Carrot [unwashed]: Carrot [washed]: Wash the carrot under running water to remove dirt. 8: ↓:↓: Peel: Peeler: Carrot [washed]: Carrot [skinless]: Use a peeler to peel the carrot. 9: ↓:↓: Remove the stem: knife, cutting board::: Carrot [peeled]: Carrot [without stem]: Use a knife to remove the stem from the carrot on the cutting board. 10: ↓:↓: Prepare the knife: knife:::: Prepare the knife. 11: ↓:↓: Prepare the cutting board: cutting board::::: Prepare the cutting board. 12: ↓:↓: Cut the ingredients: knife, cutting board::: Carrot [without stem]: Carrot [thinly sliced]: Slice the carrot thinly. 13: ↓: Slice the celery thinly. : Wash the ingredients: hands: Celery [unwashed]: Celery [washed]: Wash the celery under running water to remove any dirt. 14:↓:↓:Separate leaves and stems:Knife, cutting board:::Celery [washed]:Celery [separate leaves and stems]:Use a knife to separate the celery leaves and stems on the cutting board. 15:↓:↓:Remove strings:Knife, cutting board:::Celery [separate leaves and stems]:Celery [stringless]:Use a knife to remove the strings from the celery. 16:↓:↓:Prepare the knife:Knife::::Prepare the knife.17: ↓:↓: Prepare the cutting board: Cutting board: Prepare the cutting board. 18: ↓:↓: Cut the ingredients: Knife, cutting board: Celery [stringless]: Celery [thinly sliced]: Slice the celery thinly. 19: Put the onion, carrot, and celery from (1) and A into a pot and heat it, and remove the scum when it boils. : Prepare the pot: Pot: Prepare the pot. 20: ↓:↓: Add the ingredients: Pot: Onion, carrot, and celery [thinly sliced]: Ingredients in the pot [ready]: Put the thinly sliced ​​onion, carrot, and celery into the pot. 21: ↓:↓: Add water: Measuring cup: Water [before measuring]: Water in the pot [3 liters]: Measure the water with a measuring cup and pour it into the pot. 22: ↓:↓: Prepare the stove: Stove::::: Prepare the stove. 23: ↓:↓: Heat the pot: Stove: High heat: Until it boils: Ingredients and water in the pot [Ready]: Soup in the pot [Boiling]: Place the pot on the stove and heat it. 24: ↓:↓: Remove the scum: Skim off the scum: Once it boils: Soup in the pot [Boiling]: Soup in the pot [Scum removed]: Use a skimmer to remove the scum that rises to the surface when it boils. 25: Reduce the heat, add B, and simmer until the amount of soup is reduced to about 2 / 3. : Reduce the heat: Stove: Low heat: After the soup boils: Soup in the pot [Scum removed]: Soup in the pot [Heating on low heat]: Reduce the heat on the pot. 26: ↓:↓: Add ingredients: Hand::: Whole black peppercorns, bouquet garni, consommé, coconut flakes [Not added]: Soup in the pot [Ingredients added]: Add whole black peppercorns, bouquet garni, beef consommé, and coconut flakes to the pot. 27: ↓:↓: Reduce the soup: Pot, stove: Low heat: Until the amount of soup is reduced to 2 / 3: Soup in the pot [Ingredients added]: Soup in the pot [Reduced]: Reduce the soup until the amount of soup is reduced to 2 / 3. 28: Strain through a sieve and return to the pot. : Prepare a sieve: Sieve:::::: Prepare a sieve. 29: ↓:↓: Strain the soup: Sieve::: Soup in the pot [Reduced]: Soup in the sieve [Strained]: Strain the soup through a sieve. 30:↓:↓:Return to pot:Pot:::Soup in strainer [strained]:Soup in pot [after straining]:Return the strained soup to the pot.31: Peel apple C, remove the core, and cut it into appropriate sizes along with the onion. : Wash the apple: Hands::: Apple [unwashed]: Apple [washed]: Wash the apple under running water to remove dirt. 32: ↓:↓: Peel: Knife, cutting board::: Apple [washed]: Apple [peeled]: Use a knife to peel the apple. 33: ↓:↓: Remove the core: Knife, cutting board::: Apple [peeled]: Apple [coreless]: Use a knife to remove the core of the apple on the cutting board. 34: ↓:↓: Prepare the knife: Knife::::: Prepare the knife. 35: ↓:↓: Prepare the cutting board: Cutting board::::: Prepare the cutting board. 36: ↓:↓: Cutting ingredients: knife, cutting board::: apple [coreless]: apple [appropriate size]: cut the apple into appropriate size pieces. 37: ↓:↓: Cutting onion into appropriate size pieces: knife, cutting board::: onion [uncut]: onion [appropriate size]: cut the onion into appropriate size pieces. 38: Blend with the other C ingredients in a blender until it becomes a paste, and add to the soup from (2). : Prepare the blender: blender:::::: Prepare the blender. 39: ↓:↓: Adding ingredients: blender:: apple, onion, ginger, garlic, mango chutney, curry powder, red chili pepper, cinnamon stick [not yet added]: ingredients in the blender [ready]: put the apple, onion, ginger, garlic, mango chutney, curry powder, red chili pepper, and cinnamon stick into the blender. 40: ↓:↓: Blend in the blender: Blender::: Ingredients in the blender [Ready]: Paste in the blender [Finished]: Blend the ingredients into a paste. 41: ↓:↓: Add the paste to the soup: Hand::: Paste in the blender [Finished]: Soup in the pot [Add paste]: Add the paste to the soup. 42: Season the chicken with salt and pepper. : Season the chicken with salt and pepper: Hand::: Chicken [Cut into pieces]: Chicken [Salted and peppered]: Season the chicken with salt and pepper. 43: Heat oil in a frying pan and brown the surface of the chicken. : Prepare the frying pan: Frying pan:::: Prepare the frying pan.44: ↓:↓: Add oil: Measuring spoon: Salad oil [Not added]: Oil in the frying pan [Ready]: Measure the salad oil with a measuring spoon and put it in the frying pan. 45: ↓:↓: Heat the frying pan: Stove: Medium heat: Until the oil is warm: Oil in the frying pan [Ready]: Oil in the frying pan [Heated]: Heat the frying pan on the stove. 46: ↓:↓: Add chicken: Tongs: Chicken [Salt and peppered]: Chicken in the frying pan [Ready]: Use tongs to put the chicken into the frying pan. 47: ↓:↓: Cook the chicken: Frying pan: Medium heat: Until the surface is browned: Chicken in the frying pan [Ready]: Chicken in the frying pan [Cooked]: Brown the surface of the chicken. 48: Add to (3), cover and simmer over low heat for 30-40 minutes. :Add chicken to soup:pot:::Chicken in frying pan [cooked]:soup in pot [add chicken]:Add the cooked chicken to the soup. 49:↓:↓:Put on the lid:pot lid:::soup in pot [add chicken]:soup in pot [covered]:Put on the lid of the pot. 50:↓:↓:Simmer over low heat:stove:low heat:30-40 minutes:soup in pot [covered]:soup in pot [simmering complete]:Simmer over low heat for 30-40 minutes. 51:Add D to (4) and adjust the taste. :Add seasonings to the pot:measuring spoon:::umami stock, salt, pepper [not added]:soup in pot [seasonings added]:Measure the umami stock, salt, and pepper with a measuring spoon and add them to the pot. 52: ↓:↓: Adjust the taste: chopsticks::: soup in the pot [add seasoning]: soup in the pot [taste adjusted]: stir the soup with chopsticks to adjust the taste. 53: Cut the vegetables E into bite-sized pieces and steam them in a steamer. : Wash the carrots: hands::: carrots [unwashed]: carrots [washed]: wash the carrots under running water to remove dirt. 54: ↓:↓: Wash the broccoli: hands::: broccoli [unwashed]: broccoli [washed]: wash the broccoli under running water to remove dirt. 55: ↓:↓: Wash the potatoes: hands::: potatoes [unwashed]: potatoes [washed]: wash the potatoes under running water to remove dirt.56: Cut the carrots into bite-sized pieces: Knife, cutting board: Carrots [washed]: Carrots [bite-sized]: Cut the carrots into bite-sized pieces. 57: Cut the broccoli into bite-sized pieces: Knife, cutting board: Broccoli [washed]: Broccoli [bite-sized]: Cut the broccoli into bite-sized pieces. 58: Peel the potatoes: Peeler: Potatoes [washed]: Potatoes [skinless]: Use a peeler to peel the potatoes. 59: Cut the potatoes into bite-sized pieces: Knife, cutting board: Potatoes [skinless]: Potatoes [bite-sized]: Cut the potatoes into bite-sized pieces. 60: Prepare the steamer: Steamer: Prepare the steamer. 61: ↓:↓: Steam the vegetables: Steamer: Steam: Until the vegetables are soft: Vegetables [cut into bite-sized pieces]: Vegetables [Steamed]: Steam the vegetables in a steamer. 62: Put rice in a bowl, sprinkle with roasted almonds, add the vegetables, and pour the curry from (4) over it. : Prepare the bowl: Tray:::::: Prepare the bowl for serving. 63: ↓:↓: Serve the rice: Rice paddle::: Rice [cooked]: Rice in the bowl [served]: Use a rice paddle to put the rice in the bowl. 64: ↓:↓: Sprinkle with almonds: Hand::: Sliced ​​almonds [roasted]: Almonds in the bowl [sprinkled]: Sprinkle with almonds. 65: ↓:↓: Add the vegetables: Chopsticks::: Vegetables [steamed]: Vegetables in the bowl [served]: Add the vegetables. 66: ↓: ↓: Pour the curry: Ladle:: Curry in the pot [Finished]: Curry in the bowl [Served]: Pour the curry.

[0092] [61. Prompt given to the generating AI during the recipe detailing result evaluation process] # Instructions You are a recipe evaluation expert. Evaluate whether the given recipe is easy for cooking beginners to understand, by comparing it with the correct data as instructed. # Types of instructions and information provided - Correct answer data: Data output in tabular format as a model answer, rewritten for cooking beginners based on the recipe being evaluated. - Recipe refinement result: Data output in tabular format as the recipe being evaluated. ## Explanation of the structure of the recipe data - Row: The smallest unit of the cooking process in the recipe. - List of columns - Column 1: No.: Number assigned to each cooking process. - Column 2: Original recipe: The process of the given recipe before refinement. - Column 3: Task: The process obtained by breaking down the process of the given recipe. - Column 4: Module: Category value of the module. - Column 5: Cooking utensils: Cooking utensils used in the detailed process. - Column 6: Degree: Information such as heat level and water temperature in that cooking process. - Column 7: Timing: Cooking guideline information in that cooking process. - Column 8: Ingredient state [Before cooking process]: The state of the ingredients before performing that cooking process. (To inform the cook of the state before and after that process.) - Column 9: Ingredient state [After cooking process]: The state of the ingredients after performing that cooking process. (To inform the cook of the state before and after the process) - Column 10: Detailed process: A detailed explanation of the cooking process, which is a division of the given recipe into the smallest units. Using the information from columns 2 to 7, the smallest unit of the cooking process is explained in a way that is easy for even cooking beginners to understand. # Evaluation method Follow the steps below to determine whether it is 0 or 1 (0: the task is inconsistent with the original recipe, 1: the task is consistent with the original recipe). - Step 1: Look at the "Task" column in the correct data and the recipe detailing result. - Step 2: Compare the contents of the "Task" column in the correct data and the recipe detailing result. - Step 3: In the "Task" column of the correct data and the recipe detailing result, create pairs where the content of the tasks in each row matches (one task each from the correct data and the recipe detailing result). - Step 4: In the recipe detailing result, if there are 3 tasks (= 3 rows) where no pair is formed (there are no similar tasks in the correct data), determine it as 0. Otherwise, determine it as 1.[Correct Data] No.: Original Recipe: Task: Detailed Steps (Inserting chunks based on the original recipe | Provide only No., Original Recipe, Task, and Detailed Steps) [Recipe Refinement Result] No.: Original Recipe: Task: Detailed Steps (Inserting chunks based on the original recipe | Provide only No., Original Recipe, Task, and Detailed Steps) # Evaluation Method Follow the steps below to determine whether the detailed step description is sufficient or excessive, or whether it is adequate or excessive. - Step 1: Focus on the "Detailed Steps" column in the correct data and the recipe refinement result. - Step 2: Compare the contents of the "Detailed Steps" column in the correct data and the recipe refinement result. - Step 3: In the "Detailed Steps" column of the correct data and the recipe refinement result, create pairs of detailed steps (one detailed step from the correct data and one detailed step from the recipe refinement result) where the contents of each row match. - Step 4: In the recipe refinement results, if there are 3 detail steps (= 3 rows) where the pair is not formed (because there is no matching detail step in the correct data), judge it as 0. If this does not apply, judge it as 1. [Correct Data] No.: Original Recipe: Task: Detail Step (Chunk insertion per original recipe unit | Provide only No., Original Recipe, Task, Detail Step) [Recipe Refinement Results] No.: Original Recipe: Task: Detail Step (Chunk insertion per original recipe unit | Provide only No., Original Recipe, Task, Detail Step).

[0093] [621. Evaluation results for the first task output by the generated AI in the recipe refinement result evaluation process] The evaluation will be carried out according to the following procedure. ### Step 1 Focus on the "Task" column of the correct answer data and the recipe refinement result. ### Step 2 Compare the contents of the "Task" column of the correct answer data and the recipe refinement result. ### Step 3 Create pairs in which the contents of the tasks in each row match. - **Pair 1** - Correct answer data: Slice the onion thinly. - Recipe refinement result: Slice the onion thinly. - **Pair 2** - Correct answer data: Slice the carrot thinly. - Recipe refinement result: Slice the carrot thinly. - **Pair 3** - Correct answer data: Slice the celery thinly. - Recipe refinement result: Slice the celery thinly. ### Step 4 Check if there are 3 tasks (= 3 rows) in the recipe refinement result for which no pairs are formed. - There are 3 pairs, and all tasks match, so there are no tasks that do not form a pair. ### Evaluation Results - Result: 1 (Task is consistent with the original recipe) Based on the above evaluation, it was determined that the task is consistent with the original recipe.

[0094] [622. Evaluation results for the second task output by the generated AI in the recipe detailing result evaluation process] The evaluation will be carried out according to the following procedure. ### Step 1: Focus on the "Task" column of the correct answer data and the recipe detailing result. ### Step 2: Compare the contents of the "Task" column of the correct answer data and the recipe detailing result. ### Step 3: Create pairs whose contents match. #### "Task" column of the correct answer data - Put the onion, carrot, and celery from (1) and A into a pot and heat it. - ↓ - Put the thinly sliced ​​onion, carrot, and celery into the pot. - ↓ - Place the pot on the stove and heat it over high heat until it boils. - ↓ - Remove the scum once it boils. - ↓ - Reduce the heat of the pot. - ↓ - Add B to the pot. - ↓ - Simmer until the amount of soup is reduced to about 2 / 3. - ↓ - Strain the soup through a sieve. - ↓ - Return the strained soup to the pot. #### Recipe detailing result "Task" column - Put thinly sliced ​​onion, carrot, celery, shredded chicken bones, and water into a pot. - ↓ - Put thinly sliced ​​onion, carrot, celery, shredded chicken bones, and water into a pot. - ↓ - Place the pot on the stove and heat it. - ↓ - Once it boils, use a skimmer to remove the scum. - ↓ - Reduce the heat of the pot. - ↓ - Add whole black peppercorns, bouquet garni, beef consommé, and coconut flakes to the pot. - ↓ - Simmer the soup in the pot until the amount of soup is reduced to 2 / 3. - ↓ - Strain it through a sieve. - ↓ - Return the strained soup to the pot. ### Pair creation result 1. Put the onion, carrot, and celery from (1) and A into a pot and heat it. - Put thinly sliced ​​onions, carrots, celery, shredded chicken bones, and water into a pot. (Inconsistent) 2. ↓ - ↓ (Agree) 3. Place the pot on the stove and heat over high heat until it boils. - Place the pot on the stove and heat. (Inconsistent) 4. Remove the scum once it boils. - Remove the scum once it boils using a skimmer. (Agree) 5. Reduce the heat of the pot. - Reduce the heat of the pot. (Agree) 6. Add B to the pot.- Add whole black peppercorns, bouquet garni, beef consommé, and coconut flakes to the pot. (Mismatch) 7. Simmer until the soup is reduced to about 2 / 3 of its original volume. - Simmer the soup in the pot until the soup is reduced to 2 / 3 of its original volume. (Match) 8. Strain the soup through a sieve. - Strain the soup through a sieve. (Match) 9. Return the strained soup to the pot. - Return the strained soup to the pot. (Match) ### Step 4: Check the number of tasks where the pairs do not match. - Mismatched tasks: 1. Put the onion, carrot, and celery from (1) and A into a pot and heat it. (Mismatch) 2. Place the pot on the stove and heat it over high heat until it boils. (Mismatch) 3. Add B to the pot. (Mismatch) There are 3 mismatched tasks, so the evaluation is **0**.

[0095] [623. Evaluation results for the third task output by the generated AI in the recipe detailing result evaluation process] Step 1: Focus on the "Task" column of the correct answer data and the recipe detailing result. Step 2: Compare the contents of the "Task" column of the correct answer data and the recipe detailing result. | Task in correct answer data | Task in recipe detailing result | | Peel apple C and remove the core. | Peel the apple | | Wash the apple under running water to remove dirt. | Wash the apple. | | Peel the apple with a knife. | Peel the apple using a peeler. | | Remove the core of the apple with a knife. | Remove the core of the apple with a knife. | | Cut the apple into appropriate sizes. | Cut the apple into appropriate sizes using a knife on a cutting board. | | Cut the onion into appropriate sizes. | Cut the onion into appropriate sizes. | | Peel the onion by hand. | Peel the onion. | | Remove the stem of the onion with a knife. | Cut the onion into appropriate sizes using a knife on a cutting board. | | Cut the onion into appropriate sizes. | None | | Blend with the other C ingredients in a blender until it becomes a paste. | Prepare ginger, garlic, mango chutney, curry powder, red chili pepper, and cinnamon stick. | | Prepare a blender. | None | | Put the apple, onion, ginger, garlic, mango chutney, curry powder, red chili pepper, and cinnamon stick into the blender. | Put the prepared ingredients into the blender. | | Blend the ingredients into a paste. | Blend the ingredients in the blender until it becomes a paste. | | Add the paste made from C to the soup from (2). | Add the paste to the soup. | Step 3: Create pairs with matching content. - Pair 1: Correct data "Peel apple C and remove the core." - Recipe detail result "Peel the apple" - Pair 2: Correct data "Wash the apple under running water to remove dirt." - Recipe detail result "Wash the apple." - Pair 3: Correct data "Remove the core from the apple with a knife." - Recipe detail result "Remove the core from the apple with a knife." - Pair 4: Correct data "Cut the onion into appropriate sizes."- Recipe refinement result: "Cut the onion into appropriate-sized pieces." - Pair 5: Correct data: "Blend it with the other C ingredients in a blender until it becomes a paste." - Recipe refinement result: "Prepare ginger, garlic, mango chutney, curry powder, red chili peppers, and a cinnamon stick." - Pair 6: Correct data: "Prepare a blender." - Recipe refinement result: "None." - Pair 7: Correct data: "Blend the ingredients in a blender until they become a paste." - Recipe refinement result: "Blend the ingredients in a blender until they become a paste." - Pair 8: Correct data: "Add the paste made from C to the soup from (2)." - Recipe refinement result: "Add the paste made from C to the soup." Step 4: Check which tasks do not form pairs. The following tasks do not form pairs. - The tasks "Peel the apple with a knife." (Correct data) and "Peel the apple using a peeler." (Recipe refinement result) are similar but do not exactly match. - The task "Cut the onion into appropriate-sized pieces" (correct answer data) and "Cut the onion into appropriate-sized pieces using a knife on a cutting board" (recipe detail result) do not form a pair. Based on the evaluation, there are two tasks that do not form a pair, so the result is 1. Final evaluation: 1.

[0096] [624. Evaluation results for the fourth task output by the generated AI in the recipe detailing result evaluation process] Follow the procedure to perform the evaluation. ### Step 1 Focus on the "Task" column of the correct answer data and the recipe detailing result. ### Step 2 Compare the contents of the "Task" column of the correct answer data and the recipe detailing result. - Correct answer data: - 36: Season the chicken with salt and pepper. - 37: Heat oil in a frying pan. - 38: Brown the surface of the chicken. - 39: Add the browned chicken to (3). - 40: Cover and simmer over low heat for 30-40 minutes. - Recipe detailing result: - 32: Season the chicken with salt. - 33: Season the chicken with pepper. - 35: Put oil in a frying pan. - 36: Heat the frying pan on the stove. - 38: Brown the surface of the chicken. - 39: Add the roasted chicken to the soup from (3) - 41: Reduce the heat. - 42: Simmer over low heat for 30-40 minutes. ### Step 3 Create pairs. - Correct data 36 and detailed recipe results 32, 33: - Season the chicken with salt and pepper. → Season the chicken with salt, season the chicken with pepper. - Correct data 37 and detailed recipe result 35: - Heat oil in a frying pan → Put oil in a frying pan - Correct data 38 and detailed recipe result 38: - Brown the surface of the chicken. → Brown the surface of the chicken. - Correct data 39 and recipe refinement result 39: - Add the roasted chicken to (3) → Add the roasted chicken to the soup in (3) - Correct data 40 and recipe refinement results 41, 42: - Cover and simmer over low heat for 30-40 minutes → Reduce the heat., Simmer over low heat for 30-40 minutes. ### Step 4 Check for tasks that are not completed. - Correct data 37 (Heat oil in a frying pan) and recipe refinement result 35 (Put oil in a frying pan) are a perfect match and form a pair. - Correct data 40 (Cover and simmer over low heat for 30-40 minutes) and recipe refinement result 41 (Reduce the heat.) are not a match, but a pair exists with 42 (Simmer over low heat for 30-40 minutes.), so it is accepted as a pair.### Since there are fewer than 3 tasks that did not result in a conclusion, and all tasks are consistent with the original recipe, the evaluation will be "1".

[0097] [625. Evaluation results for the fifth task output by the generated AI in the recipe refinement result evaluation process] The evaluation results are as follows: Step 1: Focus on the "Task" column of the correct answer data and the recipe refinement result Step 2: Compare the contents of the "Task" column of the correct answer data and the recipe refinement result - Task of the correct answer data: 1. Add D to (4) 2. Adjust the taste 3. Prepare cooking utensils. 4. Cut the carrots into bite-sized pieces. 5. Cut the broccoli into bite-sized pieces. 6. Cut the potatoes into bite-sized pieces. 7. Steam the vegetables in a steamer. 8. Put rice in a bowl 9. Sprinkle roasted almonds on top of the rice 10. Add the vegetables 11. Pour the curry from (4) over it. - Recipe refinement results task: 1. Add "umami stock", salt and pepper to the soup from (4) 2. Stir the soup with a ladle to adjust the taste. 3. Cut the carrots into bite-sized pieces 4. Cut the broccoli into bite-sized pieces 5. Cut the potatoes into bite-sized pieces 6. Steam the cut vegetables in a steamer 7. Serve the rice in a bowl 8. Sprinkle the roasted almonds on top of the rice 9. Place the steamed vegetables next to the rice in the bowl 10. Use a ladle to pour the curry from (4) over the rice in the bowl. Step 3: Create pairs in the correct data and the recipe refinement results task where the content of each row's task matches - Pairs formed: - 1: Add D to (4) vs Add "umami stock", salt and pepper to the soup from (4) → Mismatch - 2: Adjust the taste vs Stir the soup with a ladle to adjust the taste. → Inconsistency - 3: Prepare the cooking utensils. vs Omitted (Inconsistency) - 4: Cut the carrots into bite-sized pieces. vs Cut the carrots into bite-sized pieces → Agree - 5: Cut the broccoli into bite-sized pieces. vs Cut the broccoli into bite-sized pieces → Agree - 6: Cut the potatoes into bite-sized pieces. vs Cut the potatoes into bite-sized pieces → Agree - 7: Steam the vegetables in a steamer.vs Steam the chopped vegetables in a steamer → Match - 8: Put rice in a bowl vs Serve rice in a bowl → Match - 9: Sprinkle roasted almonds over the rice vs Sprinkle roasted almonds on top of the rice → Match - 10: Add vegetables vs Place the steamed vegetables next to the rice in the bowl → Match - 11: Pour the curry from (4) over the rice. vs Use a ladle to pour the curry from (4) over the rice in the bowl. → Match Step 4: In the recipe detailing results, if there are 3 tasks (= 3 lines) where the pairs do not match, judge as 0, otherwise judge as 1 - The tasks that do not match are as follows: 1. Add D to (4) vs Add "umami stock", salt and pepper to the soup from (4) → Mismatch 2. Adjust the taste vs Stir the soup with a ladle and adjust the taste. → Mismatch 3. Prepare the cooking utensils. vs Omitted (Inconsistency) There are 3 tasks that were not completed, so the final evaluation is **0**.

[0098] [631. Evaluation result for the first detailed step output by the generated AI in the recipe detailing result evaluation process] Step 1: Focus on the "Detailed Step" column of the correct data and the recipe detailing result. Step 2: Compare the contents of the "Detailed Step" column of the correct data and the recipe detailing result. Comparing the detailed steps of the ground truth data with the detailed steps of the recipe refinement results, we find the following: - Ground truth data line 1 vs. recipe refinement result line 1: The content does not match (the ground truth data includes specific instructions for preparing tools, such as "Prepare a knife, cutting board, and peeler," while the recipe refinement result only says "Prepare an onion") - Ground truth data line 2 vs. recipe refinement result line 2: The content matches (peel the onion) - Ground truth data line 3 vs. recipe refinement result line 3: The content does not match (the ground truth data says "remove the stem of the onion using a knife," while the recipe refinement result says "slice the onion thinly on a cutting board using a knife") - Ground truth data line 4 vs. recipe refinement result line 4: The content does not match (the ground truth data does not include specific instructions for slicing) - Ground truth data line 5 vs. recipe refinement result line 5: Content does not match (The correct data has the detail, "Wash the carrots under running water to remove dirt," but the detailed recipe result is simplified to "Wash the carrots") - Correct data line 6 vs Detailed recipe result line 6: Content matches (Peel with a peeler) - Correct data line 7 vs Detailed recipe result line 7: Content does not match (The correct data says, "Remove the top of the carrot with a knife," but the detailed recipe result says, "Slice the carrot thinly on a cutting board using a knife") - Correct data line 8 vs Detailed recipe result line 8: Content does not match (The correct data does not specify how to slice thinly) - Correct data line 9 vs Detailed recipe result line 9: The content does not match (the correct answer data includes the detailed instruction "Wash the celery under running water to remove dirt," but the detailed recipe result is simplified to "Wash the celery").- Ground truth data line 10 vs. Recipe detail result line 10: Content does not match (The ground truth data says "Cut the celery leaves and stalks with a knife," but the recipe detail result says "Slice the celery thinly on a cutting board using a knife") - Ground truth data line 11 vs. Recipe detail result: No matching line (Does not exist in the ground truth data) - Ground truth data line 12 vs. Recipe detail result: No matching line (Does not exist in the ground truth data) Step 3: When pairs are made, it looks like this: - There are detailed steps for content that does not have a pair, so there are many detailed steps for which no pair is formed. Step 4: In the recipe detail result, if there are 3 detailed steps (= 3 lines) for which no pair is formed (there is no matching detailed step in the ground truth data), it is judged as 0. In this case, there is a lack of detailed steps in the recipe detail result compared to the ground truth data. Ultimately, the evaluation is 0 (there is an excess or deficiency in the explanation of the detailed steps).

[0099] [632. Evaluation results for the second detailed step output by the generated AI in the recipe detailing result evaluation process] The evaluation results are shown below. ### Step 1 We focused on the "Detailed Step" column of the correct data and the recipe detailing result. ### Step 2 We compared the contents of the "Detailed Step" column of the correct data and the recipe detailing result. ### Step 3 We created matching pairs of detailed steps. | Detailed step of correct data | Detailed step of recipe detailing result | | Prepare a pot. | Prepare a pot. | | Put thinly sliced ​​onions, carrots, and celery into the pot. | Put thinly sliced ​​onions, carrots, celery, shredded chicken bones, and water into the pot. | | Place the pot on the stove and heat over high heat until it boils. | Place the pot on the stove and heat. | | Remove the scum once it boils. | Remove the scum once it boils using a skimmer. | | Reduce the heat of the pot. | Reduce the heat of the pot. | | Add whole black peppercorns, bouquet garni, beef consommé, and coconut flakes to the pot. | Add whole black peppercorns, bouquet garni, beef consommé, and coconut flakes to the pot. | | Simmer until the soup is reduced to 2 / 3 of its original volume. | Simmer in the pot until the soup is reduced to 2 / 3 of its original volume. | | Prepare a strainer. | Strain through the strainer. | | Return the strained soup to the pot. | Return the strained soup to the pot. | ### Step 4 As a result of creating pairs, there were no detailed steps that did not exist in the ground truth data but only in the recipe detail results. Therefore, the evaluation is as follows: - All matching pairs of detailed steps exist, and there are no excesses or deficiencies. ### Evaluation Result 1: The detailed step descriptions are complete and accurate.

[0100] [633. Evaluation result for the third detailed step output by the generated AI in the recipe detailing result evaluation process] Step 1: Focus on the "Detailed Step" column of the correct data and the recipe detailing result. Step 2: Compare the contents of the "Detailed Step" column of the correct data and the recipe detailing result. - Detailed step of the correct data - Wash the apple under running water to remove dirt. - Peel the apple with a knife. - Remove the core of the apple with a knife. - Cut the apple into appropriate sizes. - Peel the onion by hand. - Remove the stem of the onion with a knife. - Cut the onion into appropriate sizes. - Prepare a blender. - Put the apple, onion, ginger, garlic, mango chutney, curry powder, red chili pepper, and cinnamon stick into the blender. - Blend the ingredients into a paste. - Add the paste to the soup. - Detailed steps of the refined recipe results - Prepare the apple. - Wash the apple. - Peel the apple using a peeler. - Remove the core from the apple with a knife. - Cut the apple into appropriate sizes on a cutting board with a knife. - Prepare the onion. - Peel the onion. - Cut the onion into appropriate sizes on a cutting board with a knife. - Prepare the ginger, garlic, mango chutney, curry powder, red chili pepper, and cinnamon stick. - Put the prepared ingredients into a blender. - Blend the ingredients in the blender until they become a paste. - Add the paste to the soup. Step 3: Create pairs of detailed steps that match between the ground truth data and the refined recipe results. - Matching detailed steps - "Wash the apple." (Correct data) <=> "Wash the apple." (Recipe detail result) - "Remove the core from the apple with a knife." (Correct data) <=> "Remove the core from the apple with a knife." (Recipe detail result) - "Cut the apple into appropriate sizes." (Correct data) <=> "Cut the apple into appropriate sizes using a knife on a cutting board." (Recipe detail result) - "Peel the onion."」 (Correct data) <=> 「Peel the onion.」 (Detailed recipe result) - 「Cut the onion into appropriate-sized pieces.」 (Correct data) <=> 「Cut the onion into appropriate-sized pieces using a knife on a cutting board.」 (Detailed recipe result) - 「Prepare ginger, garlic, mango chutney, curry powder, red chili pepper, and cinnamon stick.」 (Correct data) <=> 「Prepare ginger, garlic, mango chutney, curry powder, red chili pepper, and cinnamon stick.」 (Detailed recipe result) - 「Put the prepared ingredients into a blender.」 (Correct data) <=> 「Put the prepared ingredients into a blender.」 (Detailed recipe result) - 「Blend the ingredients into a paste.」 (Correct data) <=> 「Blend the ingredients into a paste.」 (Detailed recipe result) - 「Add the paste to the soup.」 (Correct data) <=> 「Add the paste to the soup.」 (Recipe refinement results) Step 4: In the recipe refinement results, check if there are 3 pairs (= 3 lines) of detailed steps that do not match (because there are no matching detailed steps in the ground truth data). - Different steps - "Prepare the apples." (Recipe refinement results) does not exist in the ground truth data. - "Prepare the onions." (Recipe refinement results) does not exist in the ground truth data. There are a total of 2 lines of detailed steps that do not match. Therefore, the evaluation result is 1. Evaluation result: 1 (The detailed steps are described without excess or deficiency).

[0101] [634. Evaluation result for the fourth detailed step output by the generated AI in the recipe detailing result evaluation process] Step 1: Focus on the "Detailed Step" column of the correct data and the recipe detailing result. Step 2: Compare the contents of the "Detailed Step" column of the correct data and the recipe detailing result. - When comparing the detailed steps of the correct data and the recipe detailing result, the contents are as follows: | Correct Data Detailed Step | Recipe Details Result Detailed Step | |-------------------|-------------------------| | Sprinkle an appropriate amount of salt and pepper on the chicken. | Prepare the chicken. | | ↓ | Sprinkle salt on the chicken. | | ↓ | Sprinkle pepper on the chicken. | | Prepare a frying pan. | Prepare a frying pan and add oil. | | Put it in the frying pan. | Heat the frying pan on the stove. | | Brown the surface of the chicken. | Put the chicken in the frying pan. | | Add the browned chicken to the soup. | Add the roasted chicken to the soup from (3). | | Cover the pot. | Cover the pot. | | Simmer over low heat for 30-40 minutes. | Reduce the heat. | | ↓ | Simmer over low heat for 30-40 minutes. | Step 3: Create pairs of matching detailed steps. - The matching detailed steps are as follows: 1. Season the chicken with salt and pepper. ⇔ Prepare the chicken. 2. Prepare a frying pan. ⇔ Prepare a frying pan and add oil. 3. Brown the surface of the chicken. ⇔ Place the chicken in the frying pan. 4. Add the roasted chicken to the soup. ⇔ Add the roasted chicken to the soup from (3). 5. Cover the pot. ⇔ Cover the pot. 6. Simmer over low heat for 30-40 minutes. ⇔ Simmer over low heat for 30-40 minutes. Step 4: In the recipe refinement results, if there are 3 detail steps (= 3 rows) where the pair is not valid (there are no matching detail steps in the ground truth data), it is judged as 0. - There are 2 detail steps in the recipe refinement results that do not exist in the ground truth data. - Prepare the chicken. - Season the chicken with salt.- Season the chicken with pepper. - In contrast, the correct answer data integrates these steps and does not separate them into detail. Therefore, the evaluation result is judged to be "0".

[0102] [635. Evaluation results for the fifth detailed step output by the generated AI in the recipe refinement result evaluation process] The evaluation results are shown below. ### Procedure-based evaluation - **Procedure 1**: We focused on the "Detailed Step" column of the correct answer data and the recipe refinement result. - **Procedure 2**: We compared the "Detailed Step" of both. #### Matching Detailed Step Pairs 1. Ground truth data No. 45 and recipe detail result No. 43 2. Ground truth data No. 46 and recipe detail result No. 44 3. Ground truth data No. 48 and recipe detail result No. 45 4. Ground truth data No. 49 and recipe detail result No. 46 5. Ground truth data No. 51 and recipe detail result No. 48 6. Ground truth data No. 52 and recipe detail result No. 49 7. Ground truth data No. 53 and recipe detail result No. 51 8. Ground truth data No. 54 and recipe detail result No. 52 9. Ground truth data No. 56 and recipe detail result No. 55 10. Ground truth data No. 57 and recipe detail result No. 58 11. 12. Correct Data No. 59 and Recipe Detail Result No. 60 13. Correct Data No. 61 and Recipe Detail Result No. 62 14. Correct Data No. 62 and Recipe Detail Result No. 63 #### Mismatched Detailed Steps - The recipe detail results contain three detailed steps that do not exist in the correct data, as follows: - Recipe Detail Result No. 45: "Prepare the carrots." - Recipe Detail Result No. 48: "Prepare the broccoli." - Recipe Detail Result No. 51: "Prepare the potatoes." ### Evaluation of Step 4 - Since there are three detailed steps that do not exist in the correct data, the evaluation is judged to be **0** (the explanation of the detailed steps is insufficient or excessive).

Claims

1. A control data generation device characterized by comprising: a storage unit that has access to a storage unit that stores recipe data including cooking process information; a generation unit that generates control data for controlling a cooking process based on measurement data obtained by sensors, which includes sound measurement data relating to the sound during the execution of a certain cooking process based on the recipe data, appearance measurement data relating to the appearance of the food being cooked during the execution, and the recipe data; and an output processing unit that outputs the control data generated by the generation unit.

2. The control data generation device according to claim 1, characterized in that the output processing unit outputs the control data to a drive unit that drives a cooking appliance.

3. The control data generation device according to claim 1, characterized in that the output processing unit outputs the control data to the user.

4. The control data generation device according to claim 1, characterized in that the output processing unit outputs the control data to a drive unit that drives a cooking appliance and to the user.

5. A control data generation device according to any one of claims 1 to 4, further comprising an acquisition unit that acquires the recipe data and stores it in the storage unit, wherein the generation unit generates the control data based on the sound measurement data, the appearance measurement data and the recipe data acquired by the acquisition unit.

6. The control data generation device according to claim 5, characterized in that the recipe data includes information created based on reinforcement learning.

7. The control data generation device according to claim 6, characterized in that the acquisition unit further acquires user data including information created based on reinforcement learning and stores it further in the storage unit, the sound measurement data is measurement data relating to the sound during the execution of a certain cooking process based on the recipe data and the user data, the appearance measurement data is measurement data relating to the appearance of the cooked object during the execution, and the generation unit generates the control data based on the sound measurement data and the appearance measurement data, as well as the recipe data acquired by the acquisition unit and the user data acquired by the acquisition unit.

8. The control data generation device according to claim 7, further comprising the storage unit, wherein the storage unit further stores the measurement data.

9. The control data generation device according to claim 8, characterized in that the measurement data includes information created based on reinforcement learning.

10. The control data generating device according to claim 9, characterized in that the control data is for controlling the start of cooking, the end of cooking, heating conditions, stirring conditions, or any combination thereof.

11. The control data generation device according to claim 10, characterized in that the generation unit generates the control data based on (1) the sound measurement data, the appearance measurement data, the recipe data and the user data, and (2) the resistance measurement data which is the measurement data relating to the resistance value of the cooking appliance in operation, the odor measurement data which is the measurement data relating to the odor components of the food being cooked in operation, the taste measurement data which is the measurement data relating to the taste components of the food being cooked in operation, the temperature measurement data which is the measurement data relating to the temperature of the food being cooked in operation, the weight measurement data which is the measurement data relating to the weight of the food being cooked in operation, the elasticity measurement data which is the measurement data relating to the elasticity of the food being cooked in operation, or any combination thereof.

12. The control data generation device according to any one of claims 1 to 4, characterized in that the cooking process information included in the recipe data is detailed by a generating AI that has been given a prompt to instruct it to detail the cooking process information, and the prompt includes an explanation of the concept of detailing and includes cooking utensil data describing the cooking utensils used for each ingredient and each pre-processing step, and usage data describing the uses of each cooking utensil, as auxiliary information for the detailing.

13. A control data generation method characterized in that an information processing device that can access a storage unit that stores recipe data including cooking process information performs a generation step of generating control data for controlling a cooking process based on measurement data obtained by a sensor, which includes sound measurement data relating to the sound during the execution of a certain cooking process based on the recipe data, appearance measurement data relating to the appearance of the food being cooked during the execution, and the recipe data; and an output processing step of outputting the control data generated in the generation step.

14. A control data generation program for causing an information processing device, which has access to a storage unit that stores recipe data including cooking process information, to execute: a generation step of generating control data for controlling a cooking process based on measurement data obtained by sensors, which includes sound measurement data relating to the sound during the execution of a certain cooking process based on the recipe data, appearance measurement data relating to the appearance of the food being cooked during the execution, and the recipe data; and an output processing step of outputting the control data generated in the generation step.

15. A control data generation system comprising: a sound sensor capable of measuring sound; an appearance sensor capable of measuring the appearance of an object; a storage unit for storing recipe data including cooking process information; a generation unit for generating control data for controlling a cooking process based on sound measurement data obtained by measuring with the sound sensor regarding the sound during the execution of a certain cooking process based on the recipe data, appearance measurement data obtained by measuring with the appearance sensor regarding the appearance of the object being cooked during the process, and the recipe data; and an output processing unit for outputting the control data generated by the generation unit.

16. The control data generation system according to claim 15, further comprising a cooking appliance and a drive unit for driving the cooking appliance, wherein the output processing unit outputs the control data to the drive unit.

17. The control data generation system according to claim 15, characterized in that the output processing unit outputs the control data to the user.

18. The control data generation system according to claim 15, further comprising a cooking appliance and a drive unit for driving the cooking appliance, wherein the output processing unit outputs the control data to the drive unit and the user.

19. A method for creating a trained model, characterized by creating a trained model by using training data which associates sound measurement data relating to the sound during execution of a certain cooking process based on recipe data including cooking process information, appearance measurement data relating to the appearance of the food being cooked during the process, and recipe data with control data for controlling the cooking process, and by performing a parameter training process for a machine learning model that outputs control data when sound measurement data, appearance measurement data, and recipe data are input.

20. A method for creating a model characterized by creating an adjusted model that performs the task of generating control data for controlling a cooking process by using a dataset that associates sound measurement data relating to the sound during the execution of a certain cooking process based on recipe data including cooking process information, appearance measurement data relating to the appearance of the food being cooked during the execution, and recipe data with control data for controlling the cooking process, and performing a parameter adjustment process on the base model.

21. A trained model for outputting control data to control a cooking process when measurement data obtained by a sensor, including sound measurement data relating to the sound during the execution of a cooking process based on recipe data including cooking process information, appearance measurement data relating to the appearance of the food being cooked during the execution, and recipe data are input, wherein the parameters of the trained model are learned using training data that associates sound measurement data, appearance measurement data, and recipe data with control data, and the trained model is characterized in that it functions to accept sound measurement data, appearance measurement data, and recipe data as input, perform calculations based on the parameters on the input sound measurement data, appearance measurement data, and recipe data, and output control data.

22. An adjusted model that, upon input of sound measurement data relating to the sound during the execution of a cooking process based on recipe data including cooking process information, which is measurement data obtained by measurement data measured by a sensor, appearance measurement data relating to the appearance of the food being cooked during the execution, and recipe data, performs the task of generating control data for controlling a cooking process, wherein the parameters of the adjusted model are parameters of a base model, adjusted using a dataset that associates sound measurement data, appearance measurement data, and recipe data with control data, and the adjusted model is characterized in that it causes a computer to function to accept sound measurement data, appearance measurement data, and recipe data as input, perform calculations based on the parameters on the input sound measurement data, appearance measurement data, and recipe data, and perform the task of generating control data.

23. A control data generation method characterized in that an information processing device that can access a storage unit that stores recipe data including cooking process information performs a generation step of generating control data for controlling a cooking process using a machine learning model based on measurement data obtained by measuring with a sensor, which includes sound measurement data relating to the sound during the execution of a certain cooking process based on the recipe data, appearance measurement data relating to the appearance of the food being cooked during the execution, and the recipe data, wherein the machine learning model is a trained model created by the creation method described in claim 19.

24. A control data generation method characterized in that an information processing device that can access a storage unit that stores recipe data including cooking process information performs a generation step of generating control data for controlling a cooking process using a base model based on measurement data obtained by a sensor, which includes sound measurement data relating to the sound during the execution of a certain cooking process based on the recipe data, appearance measurement data relating to the appearance of the food being cooked during the execution, and the recipe data, wherein the base model is an adjusted model created by the creation method described in claim 20.

Images