Determining degree of cooking of food

Abstract

The invention relates to a method (S1-S12) for determining the degree of cooking of food (G) processed in a household cooking appliance (1), in which at least one image (S4) of the food (G) is recorded by means of a cooking chamber camera (6), a regression analysis algorithm based on the at least one KI model outputs from the at least one image a measure of an actual degree of cooking value representing the degree of cooking (S5) and, based on the actual degree of cooking value, at least one action (S9, S11) can be triggered. The invention also relates to a household cooking appliance (1), wherein the household cooking appliance (1) has a cooking chamber (2) and a cooking chamber camera (6) and is designed to operate the method (S1-S12). The invention can be particularly advantageously applied to ovens.

Description

Technical Field

[0001] The invention relates to a method for determining the degree of doneness of food processed in a domestic cooking appliance, wherein at least one image of the food is recorded by means of a cooking chamber camera, an algorithm based on at least one KI model outputs a measure representing an actual degree of doneness value of the degree of doneness from the at least one image, and at least one action is triggered based on the actual degree of doneness value. The invention further relates to a domestic cooking appliance, wherein the domestic cooking appliance has a cooking chamber and a cooking chamber camera and is configured to run the method. The invention can be used in particular advantageously in ovens. Background Art

[0002] EP 3 477 206 A1 discloses a cooking device comprising a cooking chamber and an imaging device for capturing an image of food goods in the chamber. The computing device may be configured to calculate parameters of the food based on the detected image, which image may be displayed on a user interface. To this end, a data processing device is in communication with a camera and comprises a software module configured to receive the captured image from the camera and to calculate the degree of browning. The user interface is configured such that the user interface displays a visual scale illustrating the degree of browning.

[0003] US2021 / 0137311A1 discloses a KI (artificial intelligence) device, which includes a cooking unit configured to cook food by applying heat; a memory configured to store a cooking level classification model for determining the degree of cooking level of food; a camera configured to detect food; and a processor configured to determine the degree of cooking level from the image of the food taken using the cooking level classification model, and determine whether the determined degree of cooking level is equal to the degree of user preference level, and if the level of the determined cooking level is equal to the level of user preference level as a result of determination, the cooking unit is controlled to end the cooking of the food. The training data used for the supervised learning of the cooking level classification model can be annotated with cooking levels, and the cooking level classification model can be trained using the annotated training data. According to the cooking degree, there are multiple cooking levels. For example, the cooking level can be classified into a first level (very light cooking), a second level (light cooking), a third level (medium cooking), a fourth level (medium to thorough cooking), a fifth level (thorough cooking) and a sixth level (very thorough cooking). For this purpose, a cooking level classification model can be trained to accurately infer the labeled cooking level from the state of the captured food image. The cooking state level classification model can determine model parameters included in the artificial neural network to minimize the cost function through supervised learning. Summary of the invention

[0004] The object of the present invention is to at least partially overcome the disadvantages of the prior art and in particular to provide improved possibilities for setting and monitoring the degree of cooking, in particular the degree of browning, using artificial intelligence methods.

[0005] This object is achieved according to the features of the independent claim. Preferred embodiments can be found in particular in the dependent claims.

[0006] This object is achieved by a method for determining the degree of cookedness of food processed in a domestic cooking appliance, wherein

[0007] - recording at least one image of the food by means of a cooking chamber camera,

[0008] - outputting a measure representing an actual degree of cooking value of the degree of cooking from at least one image based on at least one KI model or a regression analysis algorithm based on at least one KI model and

[0009] - Based on the actual cookedness degree value, at least one action is triggered or can be triggered.

[0010] Compared to US2021 / 0137311A1, for example, the regression analysis algorithm does not use cooking levels that are classified as numerically restricted, but analyzes at least one image and calculates therefrom an actual degree of cooking value in digital form, the value range of which is in principle continuous. At least one KI model has been trained to calculate the actual degree of cooking value based on at least one input image. No pre-trained levels are required, but rather it works in a continuous space or a continuous domain. This in turn leads to the following advantage: the household cooking appliance can offer the user a greater breadth for achieving cooking results without being limited to pre-trained levels. Another advantage is that the actual degree of cooking value can be used to monitor and, if necessary, control the cooking process with high accuracy before the end of the cooking time.

[0011] The degree of cooking is in particular a measure of the state that the food has reached as a result of cooking. As already indicated above, the actual degree of cooking value is a numerical value from the continuous result space of the algorithm. The actual degree of cooking value corresponds to the current degree of cooking value of the food calculated from the image.

[0012] The household cooking device can be, for example, a stove, in particular a baking oven, a steam cooking device, a microwave oven or any combination thereof, for example a baking oven with steam treatment and / or microwave function. The household cooking device can have a cooking chamber that can be loaded with food, and its in particular front loading opening can be closed by a door.

[0013] The cooking chamber camera is in particular a digital camera, in particular a color camera. The cooking chamber camera is set up and arranged to record an image of the cooking chamber, wherein in particular the food present in the cooking chamber is then also imaged in the image.

[0014] The recording of the food by means of the cooking chamber camera may include recording the food together with the surroundings of the food (e.g., cooking chamber walls, food carriers, etc.). In one refinement, at least one image fed as input to the regression analysis algorithm based on the KI model also shows the surroundings of the cooking chamber. In another refinement, only pixels or image segments of at least one image showing the food are fed to the algorithm. The image segments showing the food can be separated from the recorded image by means of methods known in principle, such as object recognition, etc.

[0015] The at least one KI model may include or be based on one KI model or a plurality of KI models.

[0016] The regression analysis algorithm based on at least one KI model can include linear regression or nonlinear regression, such as nonparametric regression, semiparametric regression or robust regression. Linear regression can advantageously be implemented and evaluated particularly simply. At least one KI model can include one or more artificial neural networks.

[0017] A regression analysis algorithm based on at least one KI model is understood in particular to be an algorithm which processes at least one input image according to at least one trained KI model and outputs a numerical value, i.e., an actual degree of cooking value, as a result. Thus, at least one KI model is trained to determine the degree of cooking from an input image. Training or learning can be performed within the scope of machine learning, in particular supervised learning. Training here refers to the ability to imitate legitimacy. Regression is in particular a learning algorithm here. For example, training can be performed based on experimentally determined data, based on data from a dedicated training database, based on simulated data, such as lightened and / or darkened measurement images, rotations, mirror images, images with reduced contrast, color shifts, etc.

[0018] The output of the actual degree of doneness value from the at least one image or based on the at least one image by the regression analysis algorithm comprises in particular that the at least one image is used as a starting point or basis or input data set for the regression analysis algorithm. This can include using pixels or pixel values ​​of the image (or its fragment) as input variables for the regression analysis algorithm. A further development is that instead of using the pixel values ​​of the image themselves, variables derived therefrom, for example so-called feature vectors, are used as input variables for the regression analysis algorithm.

[0019] At least one action can be triggered based on the actual cooking degree value, in particular including: triggering at least one action when the actual cooking degree value (for example, as an absolute value or a percentage value) or a value derived therefrom (for example, a ratio or a difference to a setpoint value) meets a certain criterion, for example, reaches a certain threshold value. Different actions can be triggered by meeting different criteria.

[0020] One embodiment is that the regression analysis algorithm outputs the actual degree of doneness value from exactly one image. In other words, the regression analysis algorithm uses exactly one image to calculate the actual degree of doneness value. This is advantageous in order to keep the computational effort low and also advantageously result in a particularly fast calculation of the actual degree of doneness.

[0021] In one embodiment, a plurality of images of the food are recorded in chronological order by means of a cooking chamber camera, and a regression analysis algorithm outputs a measure representing an actual degree of doneness value of the degree of cooking from the plurality of images. In other words, the regression analysis algorithm uses the image sequence to calculate the actual degree of doneness value. This is advantageous in order to be able to determine the actual degree of doneness particularly accurately and more robustly and less prone to errors. The image sequence can, for example, include a number n of images that were last recorded for carrying out the method (“following window”), or, for example, images recorded for carrying out the method since the beginning of the method can be used, wherein then as the cooking duration progresses, the number of images increases, for example to up to 80 images or even more images recorded at different points in time. When using a plurality of images, certain images can receive a higher weight in the calculation of the actual degree of doneness value by the regression analysis algorithm, for example, the first image in the cooking process can be weighted more highly than subsequent images.

[0022] One design solution is to check whether the actual cooking degree value has reached the rated cooking degree value, and then end the cooking process as at least one action. Thereby, the following advantages are achieved: the final cooking state of the food mapped by the rated cooking degree value can be automatically reached. An improved solution is to end the cooking process when the difference between the actual cooking degree value and the rated cooking degree value reaches a value of zero. An improved solution is that the algorithm outputs the actual cooking degree value as a percentage value of the rated cooking degree value. An improved solution is that the algorithm outputs the actual cooking degree value as a difference from the rated cooking degree value. However, the actual cooking degree value and the rated cooking degree value are not limited to percentage values, and in principle can be any value.

[0023] The desired cooked degree value can be set on the user side, for example, via a user interface of the household cooking appliance or via an application (“app”) running on a user terminal, such as a smartphone or tablet. However, if necessary, the desired cooked degree value can also be taken over from a cooking program or electronic cookbook after adjustment or modification by the user.

[0024] The setpoint degree of doneness value can in principle be set to any possible value of the actual degree of doneness value that can be output by the algorithm. One improvement is that the domestic cooking appliance is set up to only provide such setpoint degree of doneness values ​​that allow a reasonable cooking result. For example, a setpoint degree of doneness value of 0% can be excluded, because this does not correspond to any cooking process. It is also possible—for example in relation to the food—to exclude setpoint degree of doneness values ​​that do not allow a successful cooking result, for example because the food would then be burnt or have a burnt taste with a high probability.

[0025] In one embodiment, the desired cooking degree value can be set on a continuous or quasi-continuous scale. This advantageously allows a particularly high degree of variability in the selection of the desired cooking degree value. Such a scale can be a percentage scale. Such a scale can extend, for example, from 10% (keep warm) via 60% ("rare"), 70% ("medium"), 80% ("well done"), 90% ("well done") to 100% ("over") or even beyond 100%, for example in quasi-continuous steps of 1% or 5%.

[0026] One embodiment provides that the desired degree of doneness value can be set on a graduated scale, wherein the levels of the scale are predefined setting ranges of desired degree of doneness values ​​that are continuous in principle. This achieves the advantage of simplifying the input of the desired degree of doneness value. For example, such a scale can only provide the levels of 10% (keep warm) through 60% ("well done"), 70% ("rare"), 80% ("medium"), 90% ("well done") and 100% ("over"). This classification can be carried out independently of the calculated actual degree of doneness value and therefore does not require classification calculations, changes to the KI model or additional learning. In particular, in the case of this embodiment, an image or a color field corresponding to the level can be displayed instead of the numerical value of the desired degree of doneness value, or in addition.

[0027] One design is that the numbers and / or positions of the levels on the scale can be changed on the user side. This provides the following advantage, that the user himself can determine how exactly he can set the desired cooking degree value.

[0028] One embodiment is to trigger at least one action during the cooking process based on the actual cooked degree value. This achieves the advantage that even during the cooking process, events related to the actual cooked degree value can be reacted to.

[0029] In one embodiment, at least one cooking parameter (i.e., an operating parameter that influences the cooking process of the food) is changed as at least one action based on the actual degree of doneness value during the cooking process. This achieves the advantage that the cooking process itself can be adjusted according to the actual degree of doneness value. The variation of at least one cooking parameter may, for example, include:

[0030] - optional switching on and off of cooking appliance functionality, e.g. switching on and off hot air operation, switching on and off specific heating elements, introducing steam, switching on and off microwave generators, switching on and off cooking chamber ventilation, automatic door opening for cooling of the cooking chamber atmosphere, etc.;

[0031] - changing the heating power, changing the position and / or rotation speed of the rotating antenna or stirrer, etc.; etc.

[0032] One design solution is that at least one message is output to the user as at least one action based on the actual cooked degree value during the cooking process. Thereby, the user can be advantageously informed of different stages or events during the cooking process. The (one or more) information can be output purely for notification without requiring an action from the user and / or in order to prompt the user to perform an action, such as turning the food over.

[0033] One design solution is to display the cooking progress as at least one action based on the actual cooking degree value. This achieves the following advantage, namely, the cooking progress can be displayed to the user with high accuracy. The cooking progress (for example, represented as a bar graph) can be displayed, for example, on a user interface of a household cooking device and / or a user terminal device, in particular a mobile user terminal device, such as a smartphone or a tablet computer. The cooking progress can be calculated from the actual cooking value calculated by the algorithm, or can be calculated and output by the algorithm itself.

[0034] One design is to display the cooking progress as the end of the cooking time or the remaining cooking duration. This is particularly user-friendly. The end of the cooking time and / or the remaining cooking duration can thus also be calculated from the actual cooking degree value calculated by the algorithm, or can be calculated and output by the algorithm itself.

[0035] An improvement is that the user inputs the desired end of the cooking time, for example as a duration or as a time point, and the domestic cooking appliance is set up to adjust the cooking process (for example by setting the heating power) so that the cooking process ends within a certain bandwidth when the cooking time entered on the user side is reached.

[0036] One design solution is that the cooking degree value is a browning value. This achieves the advantage of using a parameter that is visually well determinable and reliably represents the cooking degree of many foods as the cooking degree value. Therefore, the algorithm outputs the actual browning degree as the actual cooking degree value. The user enters a rated browning degree as a rated cooking degree value, and is not limited to a specific number of browning degree levels. However, the cooking degree value is not limited to the browning value, but can generally include visual changes in the surface of the food related to the cooking degree. In addition to or instead of the browning value, this can also be, for example, other color changes, such as a changed green hue of broccoli, size changes, such as shrinkage of salami slices or mushrooms, a reduction in the diameter of a round pizza, etc. In particular, at least one KI model can select relevant characteristics or features and weight them.

[0037] One embodiment is that images of the food are recorded cyclically during the cooking process, for example every 30 seconds or every minute, and are fed to the regression analysis algorithm, ie, as already described above, individually or as an image sequence.

[0038] The object is also achieved by a domestic cooking appliance, wherein the domestic cooking appliance has a cooking cabinet and a cooking cabinet camera and is designed to run the method according to one of the preceding claims. The domestic cooking appliance can be designed similarly to this and vice versa and have the same advantages.

[0039] In one development, the domestic cooking appliance has a data processing device which is configured to run the method. The data processing device may be a control device of the domestic cooking appliance.

[0040] One development is that the domestic cooking appliance is configured to run the method autonomously, ie independently of the data processing capabilities of an external instance. This results in the advantage that the method can also be run without a data technology coupling of the domestic cooking appliance to an external instance.

[0041] One development is that the domestic cooking appliance can be coupled to an external instance in terms of data technology and the method can be at least partially executed on the external instance. This results in the advantage that the computing performance of the domestic cooking appliance can be kept low. The external instance can be, for example, a mobile user terminal such as a smartphone or tablet, a network server or a cloud computer. The data technology coupling can be carried out, for example, via a communication module of the domestic cooking appliance, such as a WLAN module, an Ethernet module, a Bluetooth module or the like. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The above-mentioned characteristics, features and advantages of the present invention and the manner in which these are achieved will become clearer and more clearly understandable in conjunction with the following illustrative description of embodiments, which are explained in more detail in conjunction with the accompanying drawings.

[0043] Figure 1 a simplified sketch showing a domestic appliance in the form of an oven in side view as a section; and

[0044] Figure 2 A possible embodiment of the method according to the invention is shown. DETAILED DESCRIPTION

[0045] Figure 1 A simplified sketch of a cooking device in the form of an oven 1 is shown in a side view as a sectional view. The oven 1 has a cooking chamber 2, whose front loading opening can be closed by a pivotable door 3. The oven 1 also includes a control device 4 for executing operating sequences, such as cooking and cleaning sequences, as indicated here by an electric bottom heating heating element 5 shown by way of example. The control device 4 is also configured, for example programmed, as a data processing device for executing the method. In particular, a regression analysis algorithm based on at least one KI model is implemented in the control device 4.

[0046] A cooking cabinet camera 6 in the form of a digital color camera is connected to the control device 4 in terms of data technology and can record images from the cooking cabinet 2 and transmit them to the control device 4 .

[0047] The oven 1 also has a user interface in the form of a control panel 7 which has, in particular, a touch-sensitive screen (“touch screen”) on which, in a development, images recorded by the camera 6 or fragments thereof can be displayed, for example showing only the food G present in the cooking chamber 2 .

[0048] The oven 1 may also be equipped with a communication module 8, through which the data of the control device 4 and possibly captured images may be transmitted to an external instance, such as a user terminal device, in particular to a mobile user terminal device, such as a smartphone P or a tablet computer, and / or to a network server N or a cloud computer. The communication module 8 may include, for example, an Ethernet module, a WLAN module, a Bluetooth module, etc.

[0049] Figure 2 A possible embodiment of the method according to the invention is shown, which is described in more detail based on a cooking oven 1. In this exemplary embodiment, the actual current degree of cookedness is calculated from exactly one image, but in principle the calculation of the actual current degree of cookedness can also be performed from a sequence of images (not shown).

[0050] In step S1 , the cooking chamber 2 is loaded with food G by a user.

[0051] In step S2 , the user selects a desired cooking degree value in the form of a desired browning degree, for example continuously, quasi-continuously or in stages, via the control panel 7 .

[0052] In step S3 , the user starts a cooking process which is controlled by means of the control device 4 , for example by energizing the bottom heating element 5 and / or other heating bodies (not shown).

[0053] In step S4 , an image of the cooking cabinet 2 showing the food G is recorded by means of the cooking cabinet camera 6 and transmitted to the control device 4 .

[0054] In step S5, the control device 4 calculates the actual browning degree as a numerical value from a continuous value range based on the regression analysis algorithm implemented therein from the image supplied as input to the regression analysis algorithm (i.e. from the entire image or only from at least one image segment, for example only showing food).

[0055] In step S6 , the control device 4 calculates the remaining cooking time and / or the end of the cooking time from the actual browning degree and the target browning degree.

[0056] In step S7 , the ratio of the actual browning degree to the setpoint browning degree, for example in percentage, the remaining cooking time and / or the end of the cooking time are displayed to the user, for example on the control panel 7 or on the smartphone P.

[0057] In step S8, the control device 4 checks whether the actual browning degree has reached or exceeded the setpoint browning degree. If so ("Yes (J)"), the cooking process is terminated in step S9 and a corresponding message is output to the user, for example via the control panel 7 or smartphone P, if necessary.

[0058] If this is not the case (“No (N)”), the control device 4 checks in step S10 whether another criterion based on the actual browning degree is met. If this criterion is met (“Yes”), at least one action belonging to this criterion is triggered in step S11, such as changing at least one cooking parameter, outputting at least one message to the user, such as “Attention: Only 10 minutes left until the end of the cooking time” or “Please turn the food over”, etc.

[0059] After a negative result ("No") of the check in step S10 and after step S11, a check is made in step S12 whether a predetermined delay time has elapsed since the last image recording. If this is not the case ("No"), the check is continued. However, if this is the case ("Yes"), the process branches back to step S4.

[0060] Of course, the invention is not limited to the exemplary embodiments shown.

[0061] Generally speaking, “a”, “an” etc. can be understood as singular or plural, especially in the sense of “at least one” or “one or more” etc., as long as this is not explicitly excluded, for example by expressing “exactly one” etc.

[0062] Unless expressly excluded, numerical specifications may also include the exact number specified as well as the customary tolerance ranges.

[0063] Reference numerals list

[0064] 1 oven

[0065] 2 Cooking room

[0066] 3 doors

[0067] 4 Control device

[0068] 5 bottom layer heating heating element

[0069] 6Cooking room camera

[0070] 7. Control Panel

[0071] 8 Communication modules

[0072] N Network Server

[0073] P Smartphone

[0074] S1-S12 method steps.

Claims

1. A method (S1-S12) for determining the degree of cooking of food (G) processed in a domestic cooking device (1), wherein - recording at least one image (S4) of the food (G) by means of a cooking chamber camera (6), - outputting a measure representing an actual degree of cooking value of the degree of cooking from at least one image based on a regression analysis algorithm of at least one KI model (S5) and - At least one action can be triggered based on the actual cooking degree value (S9, S11).

2. The method (S1-S12) according to claim 1, wherein the regression analysis algorithm outputs an actual degree of cooking value from exactly one image (S5).

3. The method according to claim 1, wherein a plurality of images of the food (G) are recorded in chronological order by means of a cooking chamber camera (6), and a regression analysis algorithm outputs an actual degree of cooking value from the plurality of images.

4. The method according to any of the preceding claims, wherein a check is made as to whether the actual cooked degree value has reached a setpoint cooked degree value (S8) and then the cooking process is ended as at least one action (S9).

5. Method (S1-S12) according to claim 4, wherein the setpoint cooking degree value is set on a continuous or quasi-continuous scale (S2).

6. The method (S1-S12) according to claim 4, wherein the desired degree of doneness value is set on a graduated scale, wherein the levels of the scale are, in principle, predefined setting ranges of continuous desired degree of doneness values ​​(S2).

7. The method (S1-S12) according to claim 6, wherein the number and / or the position of the levels on the scale can be changed on the user side.

8. Method (S1-S12) according to any of the preceding claims, wherein at least one action (S11) is triggered during the cooking process based on the actual cooked degree value (S10).

9. The method (S1-S12) according to claim 8, wherein at least one cooking parameter (S11) is changed as at least one action based on an actual degree of cooking value during the cooking process.

10. The method (S1-S12) according to any one of claims 8 and 9, wherein at least one message (S11) is output to a user as at least one action based on the actual cooked degree value during the cooking process.

11. The method (S1-S12) according to any one of claims 2 to 8, wherein a cooking progress is calculated (S6) based on an actual cooking degree value and the cooking progress is displayed (S7). 12 . The method ( S1 - S12 ) according to claim 11 , wherein the cooking progress is calculated ( S6 ) and displayed ( S7 ) as the end of the cooking time or the remaining cooking duration.

13. The method (S1-S12) according to any one of the preceding claims, wherein the degree of cooking value is a browning value.

14. The method (S1-S12) according to any of the preceding claims, wherein images of the food are recorded cyclically during the cooking process (S12, S4) and fed to a regression analysis algorithm (S5).

15. A domestic cooking appliance (1), wherein the domestic cooking appliance (1) has a cooking cabinet (2) and a cooking cabinet camera (6) and is designed to carry out the method (S1-S12) according to any one of the preceding claims.

Images