Heating Regulator
The cooking appliance addresses user-friendly error correction and heat setting improvements by integrating a camera, operation display, and control unit to utilize user judgments and image analysis for enhanced cooking performance.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- HITACHI GLOBAL LIFE SOLUTIONS INC
- Filing Date
- 2024-11-08
- Publication Date
- 2026-05-20
AI Technical Summary
Existing cooking appliances lack user-friendly error correction mechanisms for food identification and fail to utilize user judgments in heat control settings based on pre-cooking and post-cooking images.
A cooking appliance equipped with a camera, operation display unit, and control unit that utilizes a learned food ingredient determination model to identify food states, allows user correction of misidentifications, and adjusts heat settings through machine learning based on user judgments and image analysis.
Enhances usability by enabling easy correction of food identification errors and more appropriate heat settings, leveraging user feedback and image analysis for improved cooking performance.
Smart Images
Figure 2026083564000001_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a cooking appliance having a function of determining the state of food ingredients in an image by AI (Artificial Intelligence) using a learned food ingredient determination model.
Background Art
[0002] As a cooking appliance having a function of determining the state of food ingredients in an image by AI using a learned food ingredient determination model, a cooking appliance system described in Patent Document 1 is known.
[0003] For example, in the abstract of the same document, regarding a cooking appliance system capable of improving convenience, "The cooking appliance system of the embodiment has a housing, a heating unit, an imaging unit, a determination unit, and a control unit. The housing includes a placement unit on which a cooking target is placed. The heating unit is provided in the housing and heats the cooking target. The imaging unit captures an image of the cooking target, and the color of the cooking target included in the image captured by the temperature of the heating unit is different. The determination unit converts the image captured by the imaging unit into a monochrome image, and determines the cooking condition of the cooking target based on the converted monochrome image and a learned model whose internal variables are adjusted by learning. The control unit controls the heating unit based on the determination result of the determination unit." There is such a description.
[0004] Also, in claim 4 of the same document, there is a description of "an information output unit that outputs the type of the cooking target determined by the second determination unit to a display unit, and a correction reception unit that can receive an input from a user for correcting the type of the cooking target when the type of the cooking target displayed on the display unit is incorrect. The cooking appliance system according to claim 3, further comprising."
[0005] Furthermore, claim 8 of the same document states, "A heating cooking system according to any one of claims 1 to 7, further comprising: an evaluation receiving unit that receives a user's evaluation of the cooking results after the end of operation; and a learning unit that performs additional learning of the learned model based on the user's evaluation received by the evaluation receiving unit." [Prior art documents] [Patent Documents]
[0006] [Patent Document 1] Japanese Patent Publication No. 2022-160322 [Overview of the project] [Problems that the invention aims to solve]
[0007] However, the heating cooking system described in Patent Document 1 had the following problems. Specifically, in the heating cooking system of claim 4 of the same document, the user is supposed to be able to correct errors in determining the type of food to be cooked, but paragraphs 0069 to 0071 and Figure 13 of the same document do not provide a concrete explanation of how the user corrects the type of food to be cooked. Furthermore, in the heating cooking system of claim 8 of the same document, the trained model is further trained based on the user's evaluation of the cooking results, but as explained in paragraphs 0037 to 0039, 0048 and Figure 7 of the same document, differences in the state of the food before cooking are not utilized in the training.
[0008] Therefore, the present invention aims to provide a cooking appliance that is easier to use, such as one that can easily correct errors in food identification by a trained model, and one that can make the heat setting more appropriate by utilizing not only the user's judgment regarding the cooking process but also the relationship between the user's judgment and the food image in machine learning for heat control. [Means for solving the problem]
[0009] To solve the above problems, the present invention provides a cooking appliance comprising: a cooking chamber having a heater for heating food ingredients; a camera outputting an image of food ingredients taken inside the cooking chamber; an operation display unit capable of inputting desired operations and displaying desired information; and a control unit that controls the heater and the operation display unit based on input information from the camera and the operation display unit, wherein the control unit comprises: a learned food ingredient determination model; an AI determination unit that determines the state of food ingredients in the food ingredient image based on the food ingredient determination model; and a display content determination unit that determines the display content of the operation display unit according to the determination result of the AI determination unit, and the display content determination unit displays a plurality of states of food ingredients determined by the AI determination unit in order of priority on the operation display unit. [Effects of the Invention]
[0010] The present invention provides a heating cooker that can easily correct errors in food identification by a trained model, and allows for more appropriate heat settings by utilizing not only user evaluations regarding the cooking process but also the relationship between user judgments and food images in machine learning for heat control, thereby improving usability. [Brief explanation of the drawing]
[0011] [Figure 1] A perspective view of a heating appliance according to one embodiment. [Figure 2] A cross-sectional view of a heating appliance according to one embodiment. [Figure 3] A cross-sectional view of a heating appliance according to one embodiment. [Figure 4] Functional block diagram of the heating appliance in Example 1. [Figure 5] Flowchart of the cooking process using the heating appliance in Example 1. [Figure 6] Functional block diagram of the heating appliance in Example 2. [Figure 7] An example of an image taken using the heating appliance of Example 2. [Figure 8] Flowchart of the cooking process using the heating appliance in Example 2. [Figure 9]A cooking process flowchart showing the details of step S7 in Figure 8. [Modes for carrying out the invention]
[0012] First, an outline of one embodiment of the cooking appliance 1 will be described using Figures 1 to 3. Figure 1 is a perspective view of the cooking appliance 1, Figure 2 is a cross-sectional view of the cooking appliance 1 at position AA in Figure 1, and Figure 3 is a cross-sectional view of the cooking appliance 1 at position BB in Figure 1. In the following, the directions of up, down, left, right, front, and back will be defined as shown in each figure.
[0013] As shown in each figure, the cooking appliance 1 comprises an induction heating unit 10 located at the top of the main body, a grill unit 20 located at the lower left of the main body, an operation display unit 30 located in the center of the top surface of the main body, and a control unit 40 located inside the main body. Each unit will be described below.
[0014] <Induction heating unit 10> The induction heating unit 10 is a unit that induction heats a metal pot placed on its top surface, and includes a heat-resistant glass top plate 11, a circular pot placement area 12 drawn on the top surface of the top plate 11, a top operating area 13 provided at the front end of the top surface of the top plate 11, a heating coil 14 that induction heats the metal pot on the pot placement area 12 when power is supplied, and an inverter circuit 15 that supplies high-frequency power to the heating coil 14. The illustrated cooking appliance 1 is a so-called IH cooking heater equipped with the induction heating unit 10, but the present invention may also be applied to cooking appliances that do not have the induction heating unit 10.
[0015] <Grille Unit 20> The grill unit 20 is a unit for grilling food ingredients placed inside the storage. It includes a door 21 that can be pulled out in the front-rear direction, a cooking chamber 22 which is a substantially rectangular parallelepiped space long in the front-rear direction, a heater 23 (upper heater 23a and lower heater 23b) that emits radiant heat when powered on, a mount 24 that is pulled out together with the door 21, a cooking pan 25 made of metal placed on the mount 24, a camera 26 provided at a position where it can photograph the food ingredients in the cooking pan 25 during cooking, a lighting 27 that illuminates the food ingredients during the camera 26's photographing, an exhaust duct 28 for exhausting the oil fumes generated from the food ingredients during cooking, and so on.
[0016] <Operation display unit 30> The operation display unit 30 is a unit that the user operates when setting the heating power or automatic cooking menu of the induction heating unit 10 or the grill unit 20, or presents desired information to the user. Specifically, it is a touch panel type liquid crystal display or the like. Note that the operation display unit 30 may be provided with a voice notification function for notifying information audibly. Also, a smartphone or tablet connected to the heating cooker 1 by short-range wireless communication or the like may be used as the operation display unit 30.
[0017] The information displayed on the operation display unit 30 is, for example, the state information of the food ingredients in the cooking chamber 22, the selectable heating power, and the automatic cooking menu before the start of cooking, and the selected heating power, the automatic cooking menu, the remaining heating time, or the image of the food ingredients during cooking photographed by the camera 26 after the start of cooking.
[0018] <Control unit 40> The control unit 40 is a unit that controls the induction heating unit 10 and the grill unit 20 based on input information from the camera 26 and the operation display unit 30, etc. Specifically, it is a computer equipped with hardware such as an arithmetic device like a CPU, a storage device like a semiconductor memory, and a communication device. And by the arithmetic device executing a program, each function described later is realized. However, hereinafter, such well-known technologies will be appropriately omitted in the description.
Example
[0019] Next, the heating appliance 1 according to Embodiment 1 of the present invention will be described with reference to Figures 4 to 6.
[0020] Figure 4 is a functional block diagram showing the main parts of the heating cooker 1 of this embodiment. As shown here, in addition to the above-described configurations, the heating cooker 1 of this embodiment includes a trained food identification model 41, an AI determination unit 42 that determines the type, quantity, size, temperature range (refrigerated, frozen), and other conditions of food in the image acquired from the camera 26 based on the food identification model 41, and a display content determination unit 43 that determines the food condition, menu, recipe, etc. to be displayed on the operation display unit 30 according to the determination result of the AI determination unit 42. If the heating cooker 1 is connected to an external computer (smartphone, tablet, server, etc.) via a network, the following processing may be performed using the food identification model 41, AI determination unit 42, and display content determination unit 43 provided by that external computer.
[0021] <Problem 41 of the food ingredient identification model> When generating the food ingredient identification model 41, the model's quality is improved by training it with a large amount of image data (training data) of various food ingredients. However, even with a high-quality food ingredient identification model 41, errors may still be present in the AI identification unit 42's results, as diverse forms can exist even within the same type of food ingredient. To give an extreme example, the AI may misidentify a fish in an image of a sardine that is clearly larger than usual as a saury, or misidentify a fish in an image of a saury that is clearly smaller than usual as a sardine. Therefore, in the former case, a cooking menu for saury might be displayed on the operation display unit 30 when a cooking menu for sardines is desired, and in the latter case, a cooking menu for sardines might be displayed on the operation display unit 30 when a cooking menu for saury is desired.
[0022] <Control flowchart of this embodiment> Therefore, in the heating cooker 1 of this embodiment, even if an inappropriate food condition is displayed on the operation display unit 30 due to a misjudgment by the AI judgment unit 42, the processing flowchart shown in Figure 5 is adopted so that the user can easily correct the food condition to the correct state. The details of each step in the said processing flowchart will be explained in order below.
[0023] First, in step S1, the user pulls out the door 21 of the grill unit 20, places the ingredients (for example, four large sardines) into the cooking pan 25 that is pulled out with the door 21, and then closes the door 21. This places the ingredients to be cooked into the cooking compartment 22.
[0024] Next, in step S2, the control unit 40 turns on the lights 27 and uses the camera 26 to photograph the food inside the kitchen 22.
[0025] In step S3, the AI judgment unit 42 uses the food judgment model 41 to determine the type, quantity, size, temperature range (refrigerated, frozen), and other conditions of the food in the food image captured by the camera 26. In this step, the AI judgment unit 42 outputs not only the most likely first-place judgment result, but also multiple judgment results with a probability of second place or lower. For example, if four sardines that are clearly larger than usual are photographed, the AI judgment unit 42 will output multiple judgment results such as "Saury, 4 fish, normal" as the first-place judgment result, "Sardines, 4 fish, large" as the second-place judgment result, and "Mackerel, 4 fish, small" as the third-place judgment result. Note that although an example showing outputting up to the third-place judgment result has been explained here, it is also possible to output four or more judgment results.
[0026] In step S4, the display content determination unit 43 displays candidates (e.g., ingredient names, cooking menu names, etc.) corresponding to each judgment result output in step S3 on the operation display unit 30 in order of priority. One possible display method here is to display the status of multiple ingredients in a ranked list.
[0027] In step S5, the display content determination unit 43 determines whether any of the candidates displayed on the operation display unit 30 have been selected by the user. If the requirements are met, the process proceeds to step S6; otherwise, the process returns to step S4 (for example, if the "Next Candidate" button displayed on the operation display unit 30 is pressed). In step S4, which is the step after this one, the next candidate with the following priority after the previously displayed candidate should be displayed.
[0028] In step S6, the control unit 40 retrieves pre-registered heat settings from the storage device for the selected candidate (e.g., ingredient name, cooking menu name). The heat settings retrieved here define the heat levels of the heaters 23 (upper heater 23a, lower heater 23b) in chronological order and also include information on the time required to complete cooking.
[0029] In step S7, the control unit 40 controls the power supply to the heaters 23 (upper heater 23a, lower heater 23b) according to the acquired heat setting, and grills the food.
[0030] According to the embodiment described above, even if the ingredient candidate determined by the AI judgment unit to be the most likely is incorrect, the operation display unit will display the next most likely candidate group, allowing the user to easily select the correct option from among these candidates and perform appropriate heating control. [Examples]
[0031] Next, the heating appliance 1 according to Embodiment 2 of the present invention will be described using Figures 6 to 9. Note that repetitive explanations of points common to Embodiment 1 will be omitted.
[0032] In step S6 of Example 1, the heater 23 is controlled based on the pre-registered heat settings for the candidate selected in step S5. However, the cooking result with the default heat settings is not necessarily suitable for the initial state of the ingredients or the user's preferences. Therefore, in the heating cooker 1 of this embodiment, the following configuration and cooking process flowchart are adopted to adjust the pre-registered heat settings to be more appropriate by using machine learning to analyze the relationship between the image taken before cooking and the user's judgment during cooking.
[0033] Figure 6 is a functional block diagram showing the main components of the heating cooker 1 of this embodiment. As shown here, in addition to the configurations described in Embodiment 1, the heating cooker 1 of this embodiment is equipped with a heat setting learning unit 44 in the control unit 40 that uses machine learning to set the heat level according to the situation based on pre-registered heat level settings. Although not shown in the figure, the heating cooker 1 of this embodiment is also equipped with a display content determination unit 43.
[0034] Figure 7 shows an example of an image taken by camera 26 during the cooking of four saury fish and displayed on the operation display unit 30. According to the pre-registered heat settings, there are "3 minutes remaining" until the end of cooking, but the AI judgment unit 42 has already determined that the cooking is complete and the fish has a good brown color. However, the AI judgment unit 42's determination of cooking completion does not always match the user's determination. Therefore, in this embodiment, the heating cooker 1 employs the following processing flowchart to entrust the final decision on the timing of cooking completion to the user, and to reflect the user's judgment result and the images of the ingredients before and after cooking in the learning of the heat settings.
[0035] Figure 8 is an example of a cooking process flowchart used by the heating appliance 1 of this embodiment, and Figure 9 is a cooking process flowchart showing the details of step S7 in Figure 8.
[0036] As is evident from the comparison between Figure 5 and Figure 8, the processes from Step S1 to Step S6 are equivalent in Example 1 and this embodiment. On the other hand, in Example 1, the process of Step S7 is limited to cooking the ingredients according to the acquired heat settings, whereas in this embodiment, the acquired heat settings are adjusted according to the situation and are performed in separate steps as shown in Figure 9. The details of each step in Figure 9 will be explained below.
[0037] First, in step S7a, the AI determination unit 42 uses the food determination model 41 to determine the browning of the food in the food image captured by the camera 26.
[0038] Next, in step S7b, the AI determination unit 42 determines whether the browning determined in step S7a corresponds to the browning of the food when it is finished cooking. If the requirements are met (for example, if the entire food has a suitable browning, as shown in the example image in Figure 7), the process proceeds to step S7c. If the requirements are not met (for example, if the food does not have a sufficient browning), the process returns to step S7a.
[0039] In step S7c, the heat setting learning unit 44 notifies the user via the operation display unit 30 that the food may be ready to cook. This notification to the user may be visual, such as flashing the display, or auditory, such as sounding a buzzer.
[0040] In step S7d, the heat setting learning unit 44 displays an image of the current ingredients on the operation display unit 30 and prompts the user to input a decision on whether or not additional heating is necessary.
[0041] In step S7e, the heat setting learning unit 44 uses machine learning to analyze the relationship between the image of the ingredients before cooking taken in step S2, the ingredient determination result in step S3, the candidate selected in step S5, the heat setting acquired in step S6, the image of the ingredients at the time of notification in step S7c, and the user judgment entered in step S7d, and adjusts the heat setting for that cooking menu as needed. Note that it is not necessary to learn the relationship between all of the elements listed here; it is sufficient to learn the relationship between the user judgment entered in step S7d and at least one other element.
[0042] For example, if the user judgment entered in step S7d is "no additional heating required," then the default cooking time set in step S6 was too long for the food image before cooking. Therefore, when cooking food under similar conditions in the future, the default cooking time set in step S7d is adjusted to a shorter time before cooking begins. On the other hand, if the user judgment entered in step S7d is "additional heating required," then the default cooking time set in step S6 is appropriate for the food image before cooking. Therefore, when cooking food under similar conditions in the future, the default cooking time set in step S7d is used as is before cooking begins.
[0043] In step S7f, the control unit 40 takes into account the user's decision in step S7d and continues cooking for an appropriate amount of time. For example, if the user's decision entered in step S7d is "no additional heating needed", the power to the heater 23 is immediately cut off. On the other hand, if the user's decision entered in step S7d is "additional heating needed", cooking continues until the predetermined cooking time obtained in step S6 has elapsed.
[0044] As described above, the heating appliance of this embodiment allows for improved usability by utilizing not only the user's judgment regarding the cooking process but also the relationship with images of the ingredients before and after cooking in machine learning for heat control, thereby enabling more appropriate heat settings. [Examples]
[0045] Next, we will describe the heating appliance 1 according to Example 3 of the present invention. Note that we will omit redundant explanations of points common to Example 2.
[0046] In Example 2, step S7d allowed the user to make a binary decision: "needed" or "not needed" for additional heating. In contrast, the present invention adopts the following configuration to allow for a variety of user judgments regarding the cooking process. Specifically, the operation display unit 30 is equipped with a recording function, allowing arbitrary user statements to be recorded at the timing of step S7d. Furthermore, by equipping the heat setting learning unit 44 with a generation AI, the intent of the recorded user statements can be utilized in machine learning for heat setting at the timing of step S7e.
[0047] The recorded user statements are then classified by the generation AI as follows, for example: (1) Recorded user statement "It's burning.": Classified as a statement requesting to reduce the default heat level or shorten the default cooking time. (2) Recorded user statement "It's undercooked.": Classified as a statement requesting to increase the default heat or extend the default cooking time. (3) Recorded user statement "Just right.": Classified as a statement requesting the maintenance of the default heat level and cooking time.
[0048] As described above, the heating appliance of this embodiment allows for machine learning based on a wider range of user judgments compared to Embodiment 2, thereby improving the heating settings. [Explanation of Symbols]
[0049] 100 Cooker 10 Induction heating unit 11 Top Plate 12 Pot placement part 13 Top operation section 14 Heating coil 20 Grill Units 21 doors 22 Cooking room 23 Heater 23a Top heater 23b Lower heater 24 mounting bases 25 Prepared Breads 26 cameras 27 Lighting 28 Exhaust duct 30 Operation display unit 40 Control Units 41. Food Ingredient Identification Model 42 AI judgment section 43 Display content determination section 44. Firepower Setting Learning Unit
Claims
1. A cooking chamber equipped with a heater for heating food ingredients, A camera that outputs images of ingredients taken inside the cooking cabinet, An operation display unit that allows input of desired operations and displays desired information, A heating appliance comprising a camera and a control unit that controls the heater and the operation display unit based on input information from the operation display unit, The control unit is A pre-trained food identification model, An AI determination unit that determines the state of the food in the food image based on the food determination model, It includes a display content determination unit that determines the display content of the operation display unit according to the determination result of the AI determination unit, The aforementioned display content determination unit is characterized in that it displays multiple states of the food ingredients determined by the AI determination unit on the operation display unit in order of priority.
2. In the heating appliance described in claim 1, The aforementioned state of the food ingredient is characterized by the type, quantity, size, temperature range, or combination thereof of the food ingredient.
3. In the heating appliance according to claim 1 or claim 2, The control unit is characterized in that, after one of the multiple states of the food displayed in order of priority on the operation display unit is selected, the heating of the food is started with a heat setting corresponding to the selection.
4. A cooking chamber equipped with a heater for heating food ingredients, A camera that outputs images of ingredients taken inside the cooking cabinet, An operation display unit that allows input of desired operations and displays desired information, A heating appliance comprising a camera and a control unit that controls the heater and the operation display unit based on input information from the operation display unit, The control unit is A pre-trained food identification model, An AI determination unit that determines the browning of the food in the food image based on the food determination model, It is equipped with a heat setting learning unit that adjusts pre-registered heat settings, The heating power setting learning unit is characterized in that, when the AI determination unit determines that the browning of the food has reached a predetermined browning, it notifies the user of this fact via the operation display unit and prompts the user to input their judgment.
5. In the heating appliance described in claim 4, The aforementioned heat setting learning unit is characterized by adjusting pre-registered heat settings by machine learning the relationship between the user's judgment and the food image.
6. In the heating appliance according to claim 4 or claim 5, The aforementioned user judgment is characterized by being a user statement regarding whether or not additional heating is necessary, or the degree of cooking.