Determining the degree of cooking of food

Abstract

The invention relates to a method (S1 - S12) for determining the degree of cooking of food (G) treated in a domestic cooking appliance (1), in which method: at least one image of the food (G) is captured (S4) by means of a cooking chamber camera (6); a regression analysis algorithm based on at least one AI model outputs an actual degree-of-cooking value representing a measure of the degree of cooking from the at least one image (S5); and at least one action can be triggered based on the actual degree-of-cooking value (S9, S11). The invention also relates to a domestic cooking appliance (1), the domestic cooking appliance (1) having a cooking chamber (2) and a cooking chamber camera (6) and being designed to run the method (S1 - S12). The invention can be particularly advantageously applied to ovens.

Description

[0001] Determining the degree of doneness of food

[0002] The invention relates to a method for determining the degree of doneness of food cooked in a household cooking appliance, in which at least one image of the food is captured using a cooking chamber camera, an algorithm based on at least one AI model outputs an actual degree of doneness value representing a measure of the degree of doneness from the at least one image, and at least one action is triggered based on the actual degree of doneness value. The invention also relates to a household cooking appliance, wherein the household cooking appliance has a cooking chamber and a cooking chamber camera and is configured to run the method. The invention is particularly advantageously applicable to ovens.

[0003] EP 3 477206 A1 discloses a cooking appliance comprising a cooking chamber and an imaging device for capturing an image of a food item within the chamber. A computing device may be configured to calculate a parameter for the food item based on the captured image, which may be displayed on a user interface. A data processing device is in communication with a camera and includes a software module configured to receive the captured image from the camera and calculate a browning level. A user interface is configured to display a visual scale indicating the browning level.

[0004] US 2021 / 0137311 A1 discloses an AI (Artificial Intelligence) device comprising a cooking unit configured to cook a food by applying heat, a memory configured to store a cooking level classification model for determining a degree of cooking level of a food, a camera configured to capture the food, and a processor configured to determine a degree of cooking level from an image of the captured food using the cooking level classification model, to determine whether the determined level of cooking level is equal to a level of a user preference class, and, if the determined level of cooking level is equal to the level of the user preference class as a result of the determination, to control the cooking unit to stop cooking the food.Training data used for supervised learning of the doneness classification model can be labeled with a doneness class, and the doneness classification model can be trained using the labeled training data. Depending on the doneness level, there are multiple doneness classes. For example, doneness levels can be classified into a first level (very lightly cooked), a second level (lightly cooked), a third level (moderately cooked), a fourth level (moderately to thoroughly cooked), a fifth level (thoroughly cooked), and a sixth level (very thoroughly cooked). The doneness classification model can be trained for the purpose of accurately inferring a labeled doneness class from the state of the captured food image.The cooking state class classification model can determine model parameters contained in an artificial neural network to minimize a cost function through supervised learning.

[0005] It is the object of the present invention to at least partially overcome the disadvantages of the prior art and in particular to provide an improved possibility for setting and monitoring a degree of cooking, in particular a degree of browning, using artificial intelligence methods.

[0006] This object is achieved according to the features of the independent claims. Preferred embodiments can be found in particular in the dependent claims.

[0007] The object is achieved by a method for determining the degree of cooking of food treated in a household cooking appliance, in which

[0008] - at least one picture of the food being cooked is taken using a cooking chamber camera,

[0009] - a regression analysis algorithm based on at least one Kl model outputs an actual cooking value representing a measure of the degree of cooking from the at least one image and

[0010] - at least one action is or can be triggered based on the actual cooking level.

[0011] The regression analysis algorithm therefore does not classify the cooking levels into numerically limited cooking classes, for example in contrast to US 2021 / 0137311 A1, but rather analyzes the at least one image and calculates an actual cooking level value in the form of a number whose value range is essentially continuous. The at least one class model has been trained to calculate the actual cooking level based on the at least one input image. It does not require pre-trained classes, but operates in a continuous space or domain. This in turn results in the advantage that the household cooking appliance can offer a user a wider range of achievable cooking results without being limited to pre-trained classes. A further advantage is that the actual cooking level value can be used to monitor and, if necessary, control a cooking process before the end of a cooking time with high precision.

[0012] The degree of doneness is, in particular, a measure of the condition of the food as a result of cooking. The actual degree of doneness is, as already indicated above, a numerical value from a continuous result space of the algorithm. The actual degree of doneness corresponds to the current degree of doneness of the food calculated from the image.

[0013] The household cooking appliance can, for example, be an oven, in particular an oven, a steamer, a microwave, or any combination thereof, e.g., an oven with a steam treatment and / or microwave function. The household cooking appliance can have a cooking chamber that can be loaded with food, the loading opening of which, in particular at the front, can be closed by a door.

[0014] The cooking chamber camera is, in particular, a digital camera, especially a color camera. The cooking chamber camera is configured and arranged to capture images of the cooking chamber, with the images then also depicting, in particular, the food being cooked in the cooking chamber.

[0015] The fact that an image of the food being cooked is taken using a cooking chamber camera can comprise the food being taken together with the surroundings of the food being cooked (e.g. a cooking chamber wall, a food carrier, etc.). It is a further development that the at least one image that is fed as input to the AI ​​model-based regression analysis algorithm also shows the surroundings of the cooking chamber. It is a further development that only pixels or image sections of the at least one image that show the food being cooked are fed to the algorithm. The separation of the image section showing the food being cooked from a recorded image can be carried out using basically known methods such as object recognition, etc. The at least one AI model can comprise or be based on one AI model or several AI models.

[0016] The regression analysis algorithm based on the at least one Cl model can comprise a linear regression or a nonlinear regression, such as a nonparametric regression, semiparametric regression, or robust regression. Linear regression is advantageously particularly easy to implement and evaluate. The at least one Cl model can comprise one or more artificial neural networks.

[0017] A regression analysis algorithm based on at least one AI model is understood to be an algorithm that processes the input image using at least one trained AI model and outputs a numerical value, namely the actual cooking level, as a result. The at least one AI model is therefore trained to determine the cooking level from the input image. Training or teaching can be carried out within the framework of machine learning, in particular supervised learning. Training refers to the ability of artificial intelligence to reproduce regularities. Regression represents the learning algorithm in particular.Training can be carried out, for example, using experimentally obtained data, using data from a dedicated training database, using simulated data such as brightened and / or darkened measurement images, rotations, reflections, images with reduced contrast, color shifts, etc.

[0018] The fact that the regression analysis algorithm outputs the actual cooking level value from or based on the at least one image comprises, in particular, that the at least one image is used as a starting point or basis or input data set for the regression analysis algorithm. This can include the pixels or pixel values ​​of the image (or a section thereof) being used as input variables for the regression analysis algorithm. It is a further development that not the pixel values ​​of the image, i.e. those but variables derived therefrom, such as a so-called feature vector, are used as input variables for the regression analysis algorithm. The fact that at least one action can be triggered based on the actual cooking level value comprises, in particular, that at least one action is triggered when the actual cooking level value (e.g., as an absolute value or percentage value) or a value derived therefrom (e.g.,A value (a ratio or difference to a target value) meets a certain criterion, e.g., a certain threshold is reached. Meeting different criteria can trigger different actions.

[0019] One embodiment allows the regression analysis algorithm to output the actual cooking level from exactly one image. In other words, the regression analysis algorithm uses exactly one image to calculate the actual cooking level. This is advantageous for keeping computing power low and also advantageously results in a particularly fast calculation of the actual cooking level.

[0020] In one embodiment, several images of the food being cooked are taken in chronological order using a cooking chamber camera, and the regression analysis algorithm outputs an actual cooking value from the multiple images, representing a measure of the degree of cooking. In other words, the regression analysis algorithm uses a sequence of images to calculate the actual cooking value. This is advantageous for determining the actual cooking degree particularly accurately and more robustly or less prone to errors. The sequence of images can, for example, comprise a number of n most recently acquired images for carrying out the process ("rolling window"), or it can, for example, use the images acquired for carrying out the process since the beginning of the process, with the number of images then increasing as the cooking time progresses, e.g., to up to 80 or even more images taken at different times.When using multiple images, certain images may be given a higher weight in the calculation of the actual cooking level by the regression analysis algorithm, e.g. the first image in a cooking process may be given a higher weight than subsequent images.

[0021] In one embodiment, a check is carried out to determine whether the actual degree of doneness has reached a target degree of doneness, and then a cooking process is ended as at least one action. This has the advantage that the final cooking state of the food, as represented by the target degree of doneness, can be reached automatically. In another embodiment, the cooking process is ended when the difference between the actual degree of doneness and the target degree of doneness reaches zero. In another embodiment, the actual degree of doneness is output by the algorithm as a percentage of the target degree of doneness. In another embodiment, the actual degree of doneness is output by the algorithm as a difference from the target degree of doneness. However, the actual degree of doneness and the target degree of doneness are not limited to percentages and can, in principle, be any value.

[0022] The target cooking level can be set by the user, for example, via a user interface on the household cooking appliance or via an application program ("app") running on a user device such as a smartphone or tablet. The target cooking level can also be taken from a cooking program or an electronic cookbook, possibly after adjustment or modification by the user.

[0023] The target cooking level can generally be set to any possible value of the actual cooking level output by the algorithm. It is a further development that household cooking appliances are designed to provide selectable target cooking levels only that allow for a reasonable cooking result. For example, a target cooking level of 0% can be excluded, as this does not correspond to any cooking process. Target cooking levels that do not allow for a successful cooking result can also be excluded—for example, depending on the type of food being cooked—because the food would then be highly likely to be scorched or burnt.

[0024] One embodiment allows the target cooking level to be set on a continuous or quasi-continuous scale. This advantageously allows for a particularly high degree of variability in selecting the target cooking level. The scale can be a percentage scale. Such a scale can range, for example, from 10% (keeping warm) to 60% ("rare"), 70% ("medium rare"), 80% ("medium"), 90% ("well done") to 100% ("well done"), or even beyond 100%, e.g., in quasi-continuous increments of 1% or 5%.

[0025] One embodiment allows the target cooking level to be set on a graduated scale, with the scale levels representing predefined setting ranges of the essentially continuous target cooking level. This simplifies the entry of the target cooking level. Such a scale can, for example, offer only the levels 10% (keeping warm) through 60% ("rare"), 70% ("medium rare"), 80% ("medium"), 90% ("well done"), and 100% ("crisp"). This classification can be implemented independently of the calculated actual cooking level and therefore does not require any classification calculation, modification, or additional training of the Kl model. Particularly with this embodiment, an image or a color field corresponding to a level can be displayed alternatively or additionally to numerical values ​​of the target cooking level.

[0026] One feature is that the number and / or position of the levels on the scale can be adjusted by the user. This offers the advantage of allowing the user to determine how precisely they want to set the desired cooking level.

[0027] One embodiment of the invention is that at least one action is triggered during a cooking process based on the actual cooking level. This provides the advantage of being able to react to an event dependent on the actual cooking level even during the cooking process.

[0028] One embodiment is that, based on the actual cooking level, at least one cooking parameter (i.e., an operating parameter that influences the cooking process of the food) is varied as the at least one action during a cooking process. This provides the advantage that the cooking process itself can also be adjusted based on the actual cooking level. Variations of the at least one cooking parameter can include, for example:

[0029] - Optional switching on and off of cooking appliance functions such as switching on and off a hot air operation, switching on and off certain heating elements, introducing steam, switching on and off a microwave generator, switching on and off a cooking chamber ventilation, automatic opening of the door to cool the cooking chamber atmosphere, etc.;

[0030] - Varying heating power, varying positions and / or rotation speeds of a rotating antenna or stirrer, etc.; etc.

[0031] One embodiment provides that, based on the actual cooking level during a cooking process, at least one message is output to a user as the at least one action. This advantageously allows the user to be informed about different phases or events during the cooking process. This information can be output purely for information purposes, without requiring any action from the user, and / or to prompt a user to perform an action, e.g., turning the food.

[0032] In one embodiment, a cooking progress is displayed as the at least one action based on the actual cooking level. This provides the advantage that the cooking progress can be presented to a user with high accuracy. The cooking progress (e.g., displayed as a bar chart) can be displayed, for example, on a user interface of the household cooking appliance and / or on a user device, in particular a mobile user device such as a smartphone or tablet. The cooking progress can be calculated from the actual cooking level calculated by the algorithm, or it can be calculated and output by the algorithm itself.

[0033] One feature is that the cooking progress is displayed as the end of cooking time and / or the remaining cooking time. This is particularly user-friendly. The end of cooking time and / or the remaining cooking time can also be calculated from the actual cooking level value calculated by the algorithm, or can be calculated and displayed by the algorithm itself.

[0034] It is a further development that a user enters a desired cooking time end, e.g. as a duration or as a point in time, and the household cooking appliance is set up to adapt the cooking process in such a way (e.g. by setting a heating output) that the cooking process is ended when the cooking time end entered by the user is reached, possibly within a certain range.

[0035] In one embodiment, the degree of doneness value is a browning value. This has the advantage that a parameter that is easily determined visually and reliably represents the degree of doneness of many foods is used as the degree of doneness value. The algorithm therefore outputs an actual browning degree as the actual degree of doneness value. A user enters a target browning degree as the target degree of doneness value and is not limited to a specific number of browning degree classes. However, the degree of doneness value is not limited to the browning value, but can generally include visual changes to the surface of the food that correlate with the degree of doneness. In addition to or alternatively to the browning value, these can also be other color changes, e.g. a changed green tone of broccoli, size changes such as shrinkage of salami slices or mushrooms, etc., a reduction in the diameter of a round pizza, etc.In particular, the at least one Kl model can select and weight the relevant properties or features.

[0036] It is an embodiment that during a cooking process, images of the food are taken cyclically, e.g. every 30 seconds or every minute, and fed to the regression analysis algorithm, either individually or as a sequence of images, as already described above.

[0037] The object is also achieved by a household cooking appliance, wherein the household cooking appliance has a cooking chamber and a cooking chamber camera and is configured to execute the method according to one of the preceding claims. The household cooking appliance can be designed analogously to the cooking chamber, and vice versa, and has the same advantages.

[0038] In a further development, the household cooking appliance comprises a data processing device configured to execute the method. The data processing device can be a control device of the household cooking appliance.

[0039] It is a further development that the household cooking appliance is configured to run the process autonomously, i.e., without recourse to the data processing capacity of an external entity. This offers the advantage that the process can run even without a data connection between the household cooking appliance and the external entity.

[0040] In a further development, the household cooking appliance can be data-linked to an external instance, and the method can be run at least partially on the external instance. This offers the advantage that the computing power of the household cooking appliance can be kept low. The external instance can be, for example, a mobile user device such as a smartphone or tablet, a network server, or a cloud computer. The data-linking can be established, for example, via a communication module of the household cooking appliance such as a WLAN module, an Ethernet module, a Bluetooth module, etc. The properties, features, and advantages of this invention described above, as well as the manner in which they are achieved, will become clearer and more clearly understandable in connection with the following schematic description of an exemplary embodiment, which is explained in more detail in connection with the drawings.

[0041] Fig.1 shows a simplified sketch of a household appliance in the form of an oven as a sectional side view; and

[0042] Fig.2 shows a possible embodiment of the method according to the invention.

[0043] Fig. 1 shows a simplified sketch of a cooking appliance in the form of an oven 1 as a sectional side view. The oven 1 has a cooking chamber 2 whose front loading opening can be closed by a pivoting door 3. The oven 1 further has a control device 4 for carrying out operating sequences such as cooking and cleaning sequences, as indicated here by an electric bottom heat element 5 shown as an example. The control device 4 is also configured, e.g., programmed, as a data processing device for carrying out the method. In particular, the regression analysis algorithm based on at least one Kl model is implemented in the control device 4.

[0044] A cooking chamber camera 6 in the form of a digital color camera is connected to the control device 4 for data purposes and can record images from the cooking chamber 2 and transmit them to the control device 4.

[0045] The oven 1 also has a user interface in the form of a control panel 7 with a particularly touch-sensitive screen ("touchscreen"), on which, in a further development, images taken by the camera 6 or sections thereof, which, for example, only show food G present in the cooking chamber 2, can be displayed.

[0046] The baking oven 1 can also be equipped with a communication module 8, via which data from the control device 4 and any recorded images can be transmitted to an external instance such as a user terminal, in particular to a mobile user terminal such as a smartphone P or tablet, etc., and / or to a network server N or cloud computer. The communication module 8 can, for example, comprise an Ethernet module, a WLAN module, a Bluetooth module, etc. Fig. 2 shows a possible embodiment of the method according to the invention, which is described in more detail with reference to the baking oven 1. In this exemplary embodiment, the current actual degree of cooking is calculated based on exactly one image, although in principle the current actual degree of cooking can also be calculated based on a sequence of images (not shown).

[0047] In a step S1, the cooking chamber 2 is loaded with food G by a user.

[0048] In a step S2, a user selects a target cooking level in the form of a target browning level via the control panel 7, e.g. continuously or quasi-continuously or in steps.

[0049] In a step S3, the user starts a cooking process which is controlled by means of the control device 4, e.g. by energizing the bottom heat element 5 and / or other heat elements (not shown).

[0050] In a step S4, an image of the cooking chamber 2, which shows the food G, is taken by means of the cooking chamber camera 6 and transmitted to the control device 4.

[0051] In a step S5, the control device 4 calculates a numerical actual browning degree from a continuous range of values ​​from the image supplied to it as input (ie, from the entire image or only from at least one image section, e.g. showing only the food to be cooked).

[0052] In a step S6, the control device 4 calculates a remaining cooking time and / or an end of cooking time from the actual degree of browning and the target degree of browning.

[0053] In step S7, the ratio (e.g., percentage) between the actual browning level and the target browning level, the remaining cooking time, and / or the end of cooking time are displayed to the user, e.g., on the control panel 7 or on the smartphone P. In step S8, the control device 4 checks whether the actual browning level has reached or exceeded the target browning level. If this is the case ("Yes"), the cooking process is terminated in step S9, and if necessary, a corresponding message is output to the user, e.g., via the control panel 7 or the smartphone P.

[0054] If this is not the case ("N"), the control device 4 checks in step S10 whether another criterion based on the actual degree of browning is met. If the criterion is met ("Y"), at least one action associated with this criterion is triggered in step S11, e.g., at least one cooking parameter is varied, at least one message is output to the user, such as "Attention: only 10 minutes left until end of cooking time" or "Please turn food over," etc.

[0055] Following a negative result of the check ("N") in step S10 and after step S11, a check is performed in step S12 to determine whether a predefined delay time has already elapsed since the last image was captured. If this is not the case ("N"), the check continues. If this is the case ("Y"), however, the system branches back to step S4.

[0056] Of course, the present invention is not limited to the embodiment shown.

[0057] In general, "a", "an", etc., can be understood as a singular or a plural, in particular in the sense of "at least one" or "one or more", etc., unless this is explicitly excluded, e.g. by the expression "exactly one", etc.

[0058] A numerical specification may also include the exact number specified as well as a usual tolerance range, as long as this is not explicitly excluded.

[0059] 1 oven

[0060] 2 Cooking chamber 3 Door

[0061] 4 Control device

[0062] 5 bottom heat radiators

[0063] 6 Cooking chamber camera

[0064] 7 Control panel 8 Communication module

[0065] N network server

[0066] P Smartphone

[0067] S1-S12 process steps

Claims

Patent claims 1. Method (S1 - S12) for determining a degree of cooking of food (G) treated in a household cooking appliance (1), in which - at least one image of the food (G) is taken by means of a cooking chamber camera (6) (S4), - a regression analysis algorithm based on at least one Kl model outputs an actual cooking value representing a measure of the degree of cooking from the at least one image (S5) and - based on the actual cooking level, at least one action can be triggered (S9, S11).

2. Method (S1 - S12) according to claim 1, wherein the regression analysis algorithm outputs the actual cooking level value from exactly one image (S5).

3. Method according to claim 1, wherein a plurality of images of the food to be cooked (G) are taken in chronological sequence by means of a cooking chamber camera (6) and the regression analysis algorithm outputs the actual degree of cooking value from the plurality of images.

4. Method according to one of the preceding claims, in which it is checked whether the actual degree of cooking has reached a target degree of cooking (S8), and then as at least one action a cooking process is terminated (S9).

5. Method (S1 - S12) according to claim 4, wherein the desired degree of cooking is set on a continuous or quasi-continuous scale (S2).

6. Method (S1 - S12) according to claim 4, wherein the desired degree of cooking is set on a stepped scale, the steps of the scale being predetermined setting ranges of the essentially continuous desired degree of cooking (S2).

7. Method (S1 - S12) according to claim 6, wherein the number and / or position of the steps on the scale can be varied by the user.

8. Method (S1-S12) according to one of the preceding claims, in which at least one action is triggered during a cooking process (S11) based on the actual cooking degree value (S10).

9. Method (S1-S12) according to claim 8, wherein, based on the actual degree of cooking value during a cooking process, at least one cooking parameter is varied as the at least one action (S11).

10. Method (S1 - S12) according to one of claims 8 and 9, wherein, based on the actual cooking level value during a cooking process, at least one message is output to a user as the at least one action (S11).

11. Method (S1 - S12) according to one of claims 2 to 8, in which a cooking progress is calculated (S6) and displayed (S7) based on the actual cooking degree value.

12. Method (S1 - S12) according to claim 11, wherein the cooking progress is calculated (S6) and displayed (S7) as the end of cooking time or remaining cooking time.

13. Method (S1 - S12) according to one of the preceding claims, wherein the degree of cooking value is a browning value.

14. Method (S1 - S12) according to one of the preceding claims, in which images of the food to be cooked are cyclically taken (S12, S4) during a cooking process and fed to the regression analysis algorithm (S5).

15. Household cooking appliance (1), wherein the household cooking appliance (1) has a cooking chamber (2) and a cooking chamber camera (6) and is configured to carry out the method (S1 - S12) according to one of the preceding claims.

Images