Cooking appliance and method for operating a cooking appliance

Abstract

The invention relates to a cooking appliance (2) for cooking a food item to be cooked, the cooking appliance having an optical sensor (22), a control and computing device (24) and a memory device (26), the optical sensor (22) being configured to capture an image of a food item to be cooked that can be placed in or on the cooking appliance (2), and the control and computing device (24) being configured to carry out the following steps: extracting features of the food item to be cooked from the captured image; comparing the extracted features with features of a standard cooking program food item, and determining a first comparison result on the basis thereof; as a function of the first comparison result, comparing the extracted features with features of a food item to be cooked from a food item history and, on the basis thereof, determining a second comparison result; operating the cooking appliance (2) to cook the food item to be cooked as a function of the first comparison result or as a function of the second comparison result; and storing the captured image in a user-specific database of the memory device (26) in order to update the user-specific food item history.

Description

[0001] Description

[0002] Cooking appliance and method for operating a cooking appliance

[0003] The invention relates to a cooking device and a method for operating such a cooking device for cooking food and for updating a user-specific cooking history.

[0004] In modern kitchen and household appliance design, innovative control methods are becoming increasingly important, especially in the automation and optimization of fermentation processes.

[0005] One example of this is the use of image processing technologies to control ovens and other cooking appliances, enabling precise and efficient food preparation. These methods utilize images captured of the food in the respective cooking chambers to control the cooking processes, at least partially, automatically.

[0006] This not only improves the quality of prepared food, but also optimizes energy consumption and increases user comfort. The integration of such technologies therefore represents a step forward in kitchen automation.

[0007] Operating procedures, in particular procedures for controlling an oven or a cooking appliance, are already known from documents JP 02-122119 A, EP 0 563 698 A2, DE 10 2007 048 834 A1 , EP 2662628 B1 and US 2007 / 0029306 A1.

[0008] Based on the prior art, it is an object of the invention to provide an improved cooking appliance and an improved method for operating such a cooking appliance.

[0009] According to the invention, the problem is solved by a cooking device with the features of claim 1. According to the invention, the problem is solved by a cooking device with the features of claim 5. According to the invention, the problem is further solved by a method with the features of claim 15. Advantageous embodiments and further developments of the invention are set forth in the following dependent claims.

[0010] According to a first aspect, a cooking device for cooking food is proposed. The cooking device comprises an optical sensor, a control and computing unit, and a storage unit, wherein the optical sensor is configured to capture an image of food that can be placed in or on the cooking device, and wherein the control and computing unit is configured to perform the following steps: extracting features of the food from the captured image, preferably by means of an image feature extraction algorithm; comparing the extracted features with features of at least one standard cooking program food pre-stored in the storage unit, and based on this, determining a first comparison result;Depending on the first comparison result, compare the extracted features with features of at least one food item in a user-specific food item history stored in the storage device and determine a second comparison result based on this; operate the cooking device to cook the food item depending on the first comparison result or depending on the second comparison result; and store at least the captured image in a user-specific database of the storage device to update the user-specific food item history.

[0011] According to the invention, it is also possible to "override" standard programs in a user-specific manner. Furthermore, comparisons could be made using representatives of user-specific cooking clusters / classes, rather than always using the entire user-specific cooking history.

[0012] According to a second aspect, a method for operating a cooking appliance for cooking food and updating a user-specific cooking history is proposed. The method comprises at least the following steps: placing the food in or on the cooking appliance; capturing an image of the food using an optical sensor; extracting features of the food from the captured image, in particular using an image feature extraction algorithm; comparing the extracted features with features of at least one pre-stored standard cooking program food, and based on this, determining a first comparison result; depending on the first comparison result, comparing the extracted features with features of at least one food in a user-specific cooking history and, based on this, determining a second comparison result.Operating the cooking appliance to cook the food depending on the first comparison result or depending on the second comparison result; and storing at least the captured image in a user-specific database to update the user-specific cooking history.

[0013] It is understood that the aforementioned steps, as well as any further optional steps, do not necessarily have to be carried out in the order shown, but can also be carried out in a different order. Furthermore, additional intermediate steps may be provided. The steps may also comprise one or more sub-steps without thereby departing from the scope of the claimed method. The descriptions made for the cooking appliance according to the first aspect, and the features and advantages mentioned in this context, apply equally to the method according to the second aspect, without being stated redundantly. The descriptions made for the method apply accordingly to the cooking appliance. It is understood that linguistic modifications of features formulated according to the method may be reformulated for the cooking appliance according to common linguistic practice, without such formulations needing to be explicitly mentioned.

[0014] In this context, a "cooking appliance" preferably describes a kitchen appliance used for cooking food. The "cooking appliance" can be a stove and / or an oven and / or a steamer or a microwave / dialog oven or a slow cooker and / or a sous-vide cooker for cooking food at precise temperatures and / or a multicooker that combines various cooking methods (e.g., boiling, frying, steaming) and / or a grill. The cooking appliance is designed to cook food, in particular foodstuffs and dishes prepared from them.

[0015] In the present context, the term "a cooking item" is understood to mean at least one cooking item. In the present context, the term "an image" is understood to mean at least one image. In the present context, the term "an optical sensor" is understood to mean at least one optical sensor.

[0016] The "optical sensor" may preferably comprise a camera and / or an infrared camera. The optical sensor may also be, or include, a time-of-flight (ToF) camera. When a user places food in or on the cooking appliance, or when food is to be placed there, the optical sensor may detect the food, preferably by taking an image of it. For example, opening and / or closing the door of the cooking appliance can trigger such an image capture. If the cooking appliance is switched off or in standby mode, opening the door can trigger the activation of the optical sensor. When the food is then placed in the cooking chamber of the appliance and the door is closed, closing the door can trigger the capture of at least one image of the food. Other triggers, such as time-dependent triggers or triggers that react to changes in the image, are also possible.

[0017] The "control and computing unit" can be integrated directly into the cooking appliance or arranged separately from it. The phrase "that the cooking appliance has the control and computing unit" is therefore to be understood as referring to a control and computing unit directly integrated into the cooking appliance or one that is merely communicatively coupled to the cooking appliance. "Communicatively coupled" means that at least bidirectional information exchange is possible. The control and computing unit can be designed as the system controller of the cooking appliance and preferably includes at least one processor. Alternatively, the control and computing unit can be provided by a cloud or a server.

[0018] The "storage device" can be integrated as non-volatile memory within the cooking appliance. Alternatively, the storage device can also be provided by a cloud or server that is communicatively linked to the cooking appliance. "Communicatively linked" in this context means that at least bidirectional information exchange is possible.

[0019] The "extraction of food features from the captured image" preferably describes a process for analyzing the captured image to identify and extract specific properties or features of the food being cooked. Feature extraction can also be preceded by image preprocessing, for example, to reduce noise in the image. Feature extraction is preferably performed using at least one image feature extraction algorithm, in particular a machine learning model. The preferred image feature extraction algorithm preferably analyzes the captured image to identify specific features of the food being cooked. These features can include various properties such as texture, color, shape, size, food carrier, quantity, weight, and / or possibly a cooking state (e.g., raw, cooked, burnt). Furthermore, features such as edges, corners, textures, and / or shapes of the food being cooked can be identified.The extracted features are preferably provided as feature vectors or descriptors for a subsequent feature comparison.

[0020] The "image feature extraction algorithm" preferably uses image processing and machine learning techniques to recognize patterns and / or features in the image. Such an image feature extraction algorithm may preferably employ a convolutional neural network (CNN). Alternatively, region-based CNNs (R-CNNs) designed for object recognition in images are possible. This includes variants such as Fast R-CNN, Faster R-CNN, and Mask R-CNN. The image feature extraction algorithm may also employ an autoencoder approach, which describes a neural network that learns unsupervised to extract features by encoding and subsequently decoding information from the image. The image feature extraction algorithm may also employ a feature pyramid network (FPN) or a generative adversarial network (GAN).The image feature extraction algorithm can also use transfer learning techniques, employing a pre-trained network (e.g., ResNet, Inception) to extract features from new image data without requiring retraining the entire network. Transformer architectures, visual, or multimodal approaches are also conceivable.

[0021] Foundation models such as SAM (segment anything model) or CLIP (Contrastive Language-Image Pre-Training) can be used. The training of the image feature extraction algorithm can also be self-supervised.

[0022] The image feature extraction algorithm, or the underlying machine learning model, can preferably be pre-trained. For example, the image feature extraction algorithm can comprise a pre-trained base model that can be factory-trained using a training dataset containing standard cooking products. During operation of the cooking appliance, the image feature extraction algorithm, or at least parts of it, such as individual heads, can be further trained, particularly continuously, based on the user-specific cooking history and the newly captured images and / or (meta) information, thereby optimizing it for user-specific, automatic cooking product recognition.

[0023] A "standard cooking program food" can be a food for which the manufacturer has pre-programmed a cooking program in the cooking appliance that delivers the best possible cooking results. Preferably, a large number of standard cooking program foods and their corresponding standard cooking program or operating program are pre-stored in the cooking appliance. The standard cooking program foods are therefore preferably pre-stored on the memory device upon delivery of the cooking appliance and are thus independent of user-specific use of the appliance. The standard cooking program food can therefore be a predefined food, in particular foodstuffs or dishes, that is predefined by the manufacturer at the factory for automatic food recognition.

[0024] Comparing the extracted features with features of at least one standard cooking program item pre-stored in the storage device preferably includes searching the database for similar features and evaluating the similarity. The features of at least one standard cooking program item are preferably stored in the database as feature vectors or descriptors. These feature vectors or descriptors can be indexed to speed up database searches. Indexing structures such as K-trees, Voronoi diagrams, hash values, or Annoy (Approximate Nearest Neighbors Oh Yeah) can be used.Weitere mögliche Indextypen umfassen “Exact Search for L2”, “Exact Search for Inner Product”, “Hierarchical Navigable Small World graph exploration”, “Inverted file with exact post-verification”, “Locality-Sensitive Hashing (binary flat index)”, “Scalar quantizer (SQ) in flat mode”, “Product quantizer (PQ) in flat mode”, “IVF and scalar quantizer”, “IVFADC (coarse quantizer+PQ on residuals)”, “IVFADC+R (same as IVFADC with re-ranking based on codes)”.

[0025] The comparison of the features of the captured image with the features in the database is preferably performed by a similarity comparison of the feature vectors or descriptors. Such a similarity comparison can be carried out by calculating a distance between the feature vector of the captured image and the feature vectors in the database. For example, a Euclidean distance, a Manhattan distance, or a cosine similarity can be used. Alternatively, such a similarity comparison can be performed via a similarity search, which describes an algorithm designed for nearest neighbor search and is used to find features in the database that are most similar to the features of the captured image. Based on the comparison or similarity search, an evaluation and output of the initial comparison result is performed.The comparison result can show a ranking of potentially similar standard cooking program products or a binary comparison result (similarity present: yes / no). Applying these steps allows for an efficient and accurate comparison of image features of the captured image with image features pre-stored in the database for the standard cooking program products.

[0026] If the initial comparison indicates that the extracted features match features of at least one standard cooking program's food, an operating program or standard cooking program can be executed to cook the food. Alternatively, if the initial comparison indicates that the extracted features match features of at least one standard cooking program's food, a standard cooking program suggestion can be displayed to the user, for example, via an output device. This suggestion can preferably be confirmed, rejected, or manipulated / modified by user input (manual or verbal), for example, via an input device. The (meta) information generated by the program suggestion and / or the user input is preferably stored in the user-specific cooking history in combination with the captured image or the features extracted from it.The information can be stored collectively in a feature vector or descriptor, or separately, but preferably with an assignment reference to each other.

[0027] If the first comparison result indicates that the extracted features do not match any of the features of the at least one standard cooking program food, a further comparison of the extracted features with features of foods stored in a user-specific cooking history is preferably carried out.

[0028] The "user-specific cooking history" is preferably a database stored in the storage device containing information about dishes prepared by the user with the cooking appliance. The user-specific cooking history can include dishes cooked using standard cooking programs, but especially also dishes cooked by the user that are not represented in the standard cooking program categories. The user-specific cooking history thus specifies the individual application and individual user needs placed on the cooking appliance. In particular, with the user's prior consent, at least the captured image or the information contained therein, preferably together with program and / or setting information such as temperature, operating mode, cooking time, and possibly other metadata (i.e., date and time), can be stored in the personal cooking history for each dish prepared by the user.Over a medium- to long-term period, the user's personal cooking history grows, particularly depending on their cooking habits. The user can access, interact with, and / or modify their personalized cooking history, preferably via a web and / or mobile application (app). The web and / or mobile application can also be used to control the cooking appliance in general.

[0029] Incorporating a user-specific cooking history into a cooking appliance offers several advantages. One of the main benefits is the ability to personalize the cooking experience. For example, the appliance recognizes dishes that are not among the standard recipes or foods. Furthermore, users can see which dishes they have prepared in the past, with which parameter settings, and whether they were satisfied with the results. By taking into account the user's preferences regarding cooking times, temperature, and desired consistency and / or browning, the appliance can deliver customized cooking results. Prepared recipes can be saved and quickly recalled, saving time and increasing convenience. In addition, the user-specific cooking history allows the appliance to learn from past cooking processes and automatically adjust the cooking parameters.This ideally leads to consistent and perfect results, as the cooking appliance can proactively optimize the parameters. Such adjustments not only increase efficiency but also reduce energy consumption, since the user's preferred settings are used to make the cooking process more efficient. This also results in time and energy savings. Incorporating the user's specific cooking history also enables extended functionalities. Based on previous cooking habits, the cooking appliance can even suggest new recipes that match the user's preferences.

[0030] It is also possible to create multiple profiles for multiple users, thus enabling a high degree of individualization for each user of the cooking appliance. For example, user-specific preferences for or during the preparation of a particular food can be taken into account and stored in the user-specific cooking history, making them accessible from there.

[0031] In this case, in addition to the captured images, further user interactions with the personal cooking history are preferably also continuously updated in the user-specific database. This database, preferably continuously updated, serves in particular as the basis for training at least part of the image feature extraction algorithm, which can be trained specifically for each user. The image feature extraction algorithm can, for example, include user-specific classification heads, which then enable the recognition of user-specific cooking items in the future, particularly through continuous updates of the cooking history. In this way, personal cooking items can also be automatically recognized, and a corresponding operating program, especially with appropriate user-specific settings, can be suggested to the user.

[0032] According to a further aspect, it is proposed that the characteristics of the at least one standard cooking program food are stored by the manufacturer in a standard cooking database on the storage device, and that the characteristics of the at least one cooking food in the user-specific cooking history can be generated by a user during operation of the cooking device, in particular continuously, based on the respective captured image, and can be stored in the user-specific database on the storage device.

[0033] This makes it possible, firstly, for manufacturers to provide certain standards for cooking specific foods, thus enabling a sufficiently good cooking experience, especially for users inexperienced in food preparation. Furthermore, by incorporating user-specific history, a high degree of personalization is possible, thereby providing experienced users with the most comprehensive cooking experience possible.

[0034] According to another aspect, it is proposed that comparing depending on the first comparison result means that the extracted features are compared with the features of at least one food item in the user-specific food history if the first comparison result indicates that the extracted features do not match, or do not match well enough, the features of the at least one standard cooking program food item.

[0035] The phrase "depending on the first comparison result" means that comparing the extracted features with features of at least one food item from a user-specific food history stored in the device preferably (only) occurs if a feature comparison of the features extracted from the captured image with the features of the standard cooking program foods shows that the food item that can be placed in or on the cooking appliance is not sufficiently similar to any of the standard cooking program foods or cannot be clearly classified as one of the standard cooking program foods. Whether or not a further comparison with features from the user-specific food history takes place preferably depends on the type and content of the first comparison result. Alternatively, a comparison with the user-specific food history can always be performed.

[0036] It should be mentioned that the comparison of the features of the captured image with the features of the cooked items included in the user-specific cooking history is preferably carried out in the same way as the first-mentioned comparison with the standard cooking program cooked items.

[0037] According to a further aspect, it is proposed that if the second comparison result indicates that the extracted features do not match features of the user-specific cooking history, the operation of the cooking appliance for cooking the food is carried out based on the second comparison result by user input, in particular manual input, and if the second comparison result indicates that the extracted features match features of the user-specific cooking history, an operating program proposal, in particular comprising a user query for its acceptance or rejection, is issued to the user based on the cooking history, and the cooking appliance is operated based on this.

[0038] The cooking appliance can be operated based on either the first or second comparison result. This means that if the first comparison shows that the food to be placed in or on the cooking appliance corresponds to a standard cooking program, the appliance can be operated using a standard cooking program associated with that food, which can potentially be further modified by the user. However, if the first comparison shows that the food to be placed in or on the cooking appliance does not correspond to a standard cooking program, the appliance can be operated based on the second comparison result. For example, a cooking program for a food item from the user's cooking history might be suggested, which can then be modified by the user.

[0039] If the second comparison result indicates that a cooking food item was found in the user's cooking history that is sufficiently similar to the food currently being cooked in or on the cooking appliance, at least one cooking program suggestion can be displayed based on this second comparison result. This suggestion is stored for the food item identified as sufficiently similar in the cooking history. The user can accept the at least one cooking program suggestion, select one if multiple suggestions are available, reject the cooking program suggestion(s), or manipulate the at least one cooking program suggestion to operate the cooking appliance. If the user rejects the at least one cooking program suggestion, operation can also be started via manual or voice input.

[0040] If the second comparison result indicates that no food was found in the user-specific cooking history that is sufficiently similar to the food in or on the cooking device, the cooking device can be operated by manual or voice user input based on this second comparison result.

[0041] According to a third aspect, a cooking device is proposed. The cooking device comprises an optical sensor, a control and computing unit, and a storage unit, wherein the optical sensor is configured to capture an image of food that can be placed in or on the cooking device, wherein the control and computing unit is configured to extract information about the food from the captured image and to store at least the extracted information in a user-specific food history in a user-specific database on the storage unit, wherein the cooking device is configured such that the user-specific food history can be manipulated by a user and / or evaluated by at least one machine learning model, in particular a classification model, in order to perform automatic food recognition, in particular continuously, based on this.to optimize and / or to provide the user with at least one new operating program suggestion for food that can be placed in or on the cooking appliance.

[0042] The cooking device according to the third aspect can be particularly preferably configured to perform the steps according to the first aspect. The control and computing device according to the third aspect can thus be configured to perform the following steps: extracting features of the food being cooked from the captured image, preferably by means of an image feature extraction algorithm; comparing the extracted features with features of at least one standard cooking program food pre-stored in the storage device, and determining a first comparison result based on this; depending on the first comparison result, comparing the extracted features with features of at least one food from a user-specific cooking history stored in the storage device, and determining a second comparison result based on this.Operating the cooking appliance to cook the food depending on the first comparison result or depending on the second comparison result; and storing at least the captured image in a user-specific database of the storage device to update the user-specific cooking history. The further explanations and advantages given in this regard also apply accordingly.

[0043] According to another aspect, it is proposed that storing at least the captured image in the user-specific database includes: storing an operating program for operating the cooking appliance to cook the food; and / or storing manual input data from a user that can be generated depending on the first comparison result and / or depending on the second comparison result for operating the cooking appliance; and / or storing metadata that can be generated depending on the first comparison result and / or depending on the second comparison result for operating the cooking appliance.

[0044] "Storing at least the captured image in a user-specific database" means that at least the captured image and / or the features extracted from it are stored. Furthermore, information related to the food being cooked, such as user input (e.g., accepting a cooking program suggestion or similar), cooking appliance settings, and / or other metadata from the feature comparison, such as a ranking of the comparison, the first and / or second comparison result, or similar information, can also be stored. This allows the user-specific database to be updated as comprehensively as possible, which improves the recognition of food being cooked and the resulting automatic food identification, thus enhancing the automatic operation of the cooking appliance.

[0045] According to a further aspect, it is proposed that the cooking appliance also has an input device configured to capture user input and provide it to the control and computing unit, wherein the control and computing unit is configured, based on the user input, to: manage and / or supplement the cooking history and / or weight and / or evaluate at least one cooking item in the cooking history, and update changes in the user-specific cooking history; and / or start an operating program included in the cooking history, in particular a user-specific one; and / or create a new operating program for operating the cooking appliance, and preferably assign program specifications and / or at least one image of the user-specific cooking history to the new operating program.

[0046] The "input device" preferably includes a screen. Alternatively, the input device can also be provided wirelessly via a user's device. The input device allows the user to manage their personalized cooking history, assign names to individual foods and / or cooking programs, group foods and / or cooking programs, delete foods and / or cooking programs, rate and / or favorite foods and / or cooking programs, and / or optionally add further images related to the foods, particularly raw and / or cooked versions. Adding further images can be done, for example, via the user's device. User input is preferably stored in the personalized cooking history, thereby updating it. This enables highly individualized customization and management of the user's personalized cooking history.This allows the system to respond to new dietary needs and / or changing eating habits of the user. It offers users the opportunity to interact with their personal cooking history and, in particular, makes it possible, through machine learning approaches, to generate suggestions for new user-specific operating programs or cooking programs for which corresponding foods can also be recognized in the future. This creates an improved user experience. The user's interactions preferably serve as the basis for creating or optimizing a user-specific model for the automatic recognition of foods, which recognizes not only standard foods but also user-specific foods or dishes. The user-specific model can be updated based on user behavior, preferably continuously.

[0047] The cooking appliance may also have an output and / or display device, for example, in the form of a screen. The output and / or display device may also be combined with the input device, for example, as a touchscreen. Alternatively, a user's own device may serve as the output and / or display device. Through the output and / or display device, the user can, for example, view statistics and / or other information about their specific cooking history, such as how often they have prepared a particular dish. The user can also directly start a cooking program for a specific food via the input device and / or the preferably combined output and / or display device, without having to compare image characteristics. The user can start a personalized operating program, preferably either from the cooking history or from the operating program list on the cooking appliance.

[0048] According to another aspect, it is proposed that the control and computing device is designed to link features and / or the image of the at least one food item stored in the user-specific food item history with a recipe provided by a user for preparing the at least one food item and to update the food item history based on this.

[0049] This makes it possible to add further (meta) information to a food item, which allows for further customization and / or optimization of the corresponding cooking program. Based on the information included in the recipe, it is also possible to adjust and / or update a cooking program stored in the food item's history. The user can also provide multiple recipes for a single food item.

[0050] According to another aspect, it is proposed that the recipe provided by the user includes a link to an internet link through which the recipe can be retrieved, and / or is provided by an image of the recipe, wherein the control and computing device is designed to extract features from the recipe using an image-to-text extraction algorithm and / or a text extraction algorithm, on the basis of which a new operating program for operating the cooking device can be created in the user-specific cooking history and on the basis of which the cooking history can be updated.

[0051] This describes a method for the automated processing of user-provided recipes to operate a cooking appliance more efficiently and in a more personalized way. The at least one recipe can be provided either via an internet link or as an image of the recipe, for example, from a magazine or a screenshot. The cooking appliance's control and processing unit is designed to extract features from the recipe using an image-to-text extraction algorithm and / or a text extraction algorithm. The extracted features preferably include relevant recipe information such as ingredients, quantities, cooking times, temperatures, and preparation instructions.Based on these characteristics, a new operating program is preferably created or an existing one updated, tailored to the specific requirements of the provided recipe. It is also possible to combine information from different recipes. The new or updated operating program is saved in the user's specific cooking history, thereby updating the cooking history. This enables continuous improvement and adaptation of the cooking processes to the user's preferences and previous cooking habits.

[0052] An image-to-text extraction algorithm, often referred to as Optical Character Recognition (OCR), specializes in extracting text information from images or scanned documents. The algorithm identifies and digitizes written content to make it usable for further processing.

[0053] A text extraction algorithm is designed to extract relevant information from structured or unstructured text. This algorithm can extract text from digital documents, web content, or databases. The text is preferably pre-processed to remove superfluous characters and prepare the data for analysis. The algorithm analyzes the text to identify patterns, keywords, entities (such as names, dates, and locations), and other relevant information. The relevant text information is then preferably extracted and presented in a structured format. The extracted text can then be further processed, if necessary, to convert it into the desired format, for example, into control commands for a cooking program.

[0054] According to a further aspect, it is proposed that the control and computing device is designed to enrich the characteristics and / or the at least one image of the at least one food item that can be linked to the recipe by means of data augmentation and to update the food item history based on this, in order to improve the automatic food item recognition and the creation of the new operating program, in particular by distinguishing it from stored food items and / or operating programs.

[0055] The control and computing unit is designed to enrich the characteristics and / or at least one image of the at least one food item that can be linked to a recipe through data augmentation. This enriched data is used to update the food item history.

[0056] It is also conceivable to generate complete images using generative models, particularly based on the recipe texts described above (possibly extracted). Alternatively, this can also be done using "variational autoencoders" or "stable diffusion" approaches.

[0057] The goal of this process is to improve the automatic recognition of cooking foods and the creation of new operating programs, particularly by differentiating them from already stored cooking foods and / or operating programs. Known data augmentation techniques can be used for this purpose. Data augmentation preferably encompasses various techniques aimed at increasing the quantity and variety of data available for machine learning without actually collecting new data. Common data augmentation methods include geometric transformations such as rotating, scaling, cropping, mirroring, and / or shifting images. Color modifications are also known and include adjustments to brightness, contrast, saturation, and / or color shifts. Another method is the addition of noise, such as Gaussian or speckle noise, to increase data variety.Combinations of multiple transformations can be used to create new variations. For example, random erasing replaces random areas of an image with noise or constant values, while affine transformations involve a combination of scaling, rotation, translation, and shearing.

[0058] According to another aspect, it is proposed that the control and computing device is designed to enrich the information through which the cooking history can be updated, at least partially, by data augmentation, and to update the cooking history based on this.

[0059] The information can preferably consist of the captured images and / or the features extracted from them. Furthermore, additional information, such as settings and / or metadata, can also be included. The points mentioned above regarding data augmentation apply accordingly here. Data augmentation thus allows additional data to be generated for the captured images, thereby expanding the training dataset, at least for the user-specific part of the feature extraction algorithm. Synthetic data generation can improve the training of the image feature extraction algorithm and / or optimize the recognition or classification results based on it.Furthermore, this allows for the use of a few-short-learning approach for training the feature extraction algorithm, specifically on data that is underrepresented in the cooking history. This is particularly beneficial because such underrepresented data can be augmented to increase the amount of data and thus improve the training. Data augmentation also enables the differentiation of potentially new cooking items, for which, for example, a new operating program or cooking program should be suggested, from already automatically recognizable cooking items. This improves the robustness of automatic cooking item recognition, especially for similar cooking items.According to a further aspect, it is proposed that the control and computing unit be trained to identify, using a machine learning classification model and based on information that can be used to update the cooking history, a potentially unknown food type and / or a potentially new operating program for the cooking appliance. It is particularly preferred that, if the information that can be used to update the cooking history meets a classification criterion with respect to the information stored in the cooking history and / or the storage device, the control and computing unit be trained to suggest the previously unknown food type and / or the new operating program to a user.

[0060] In this case, it is possible for the cooking appliance to automatically suggest a new operating program or cooking program to the user. An artificial intelligence (AI) or machine learning classification model preferably identifies potential candidate classes for new user-specific foods or dishes, particularly based on the images captured in the user's personal cooking history. The classification model preferably assesses the robustness of such a new user-specific candidate class, for example, based on the distances to other food classes in the feature space. If at least one distance to other food classes exceeds a threshold, the food in question is preferably recognized as a new food, and an operating program suggestion is generated accordingly. Such a threshold is an example of the aforementioned classification criterion.Other classification criteria can also be used. If the candidate class for a potentially new food item is sufficiently distinct from other food item classes, i.e., if it is robust enough, particularly when considering a robustness criterion, the user can be automatically presented with a suggestion for a new, user-specific operating program assigned to that food item. The user can then preferably decide whether to accept the suggestion and add the new personal operating program to the list of automatically recognizable food items. In this way, the user receives AI-based suggestions for new user-specific operating programs and can preferably specify which of their personal food items and / or dishes should be automatically recognized in the future. The robustness assessment relative to other classes can be improved through data augmentation.

[0061] According to another aspect, it is proposed that the machine learning classification model includes a distance metric by which the information used to update the food history can be compared with information stored in the food history and / or the storage device within a classification space. This distance metric enables the comparison of captured information, particularly the extracted features of the captured image and potentially other (meta) information, with pre-stored information, especially the features of the food items stored in the food history and potentially other related (meta) information, within a classification space. Specifically, this means that new data input into the model is compared with existing data points using this distance metric.The classification space is an abstract space in which all data points are positioned based on their (vectorized) features and the applied distance metric. This makes it possible to evaluate the similarity or dissimilarity between the new and the stored data, thus enabling a precise classification of the food that can be placed in or on the cooking appliance.

[0062] According to another aspect, it is proposed that the proposed cooking material and / or the proposed operating program can be modified and / or extended by the user with user-specific specifications.

[0063] This allows for a highly individualized creation and / or updating of an operating program or cooking program, in order to respond as comprehensively as possible to specific user preferences, such as degree of cooking and / or degree of browning, etc.

[0064] An embodiment of the invention is shown purely schematically in the drawings and is described in more detail below. It shows

[0065] Figure 1 shows a schematic view of an exemplary cooking appliance;

[0066] Figure 2 shows a schematic flowchart of an exemplary procedure;

[0067] Figure 3 shows a schematic flowchart of a detail of an exemplary

[0068] Procedure;

[0069] Figure 4 shows a schematic diagram of a food classification with data augmentation;

[0070] Figure 5 shows exemplary distributions of Gargut classes and class representations; and

[0071] Figure 6 is a schematic diagram showing the addition of a candidate class as a new cooking class.

[0072] Fig. 1 shows a slightly perspective overall view of a cooking appliance 2 according to the invention. The cooking appliance 2 is, in this case purely by way of example, an oven with connected cooking zones 20. The cooking zones 20 can be designed as gas-powered fireplaces, as radiant heating elements, or as induction cooktops. Naturally, the cooking appliance 2 may also have no cooking zones 20 in other embodiments. The cooking appliance 2 has an input device 21 on a front surface 18, which runs orthogonally to a side surface 17, in an upper area of ​​the cooking appliance 2. This input device 21 may, for example, comprise several control elements, which may be designed, for example, as rotary knobs or touch controls, and which may be used to set an operating mode and / or at least one temperature of the cooking appliance 2.The upper section may also include a display to show the operating status, current state, target state, and / or deviations, as well as the further procedure for cooking food using the cooking device 2. The input device 21 may also be in the form of a touchscreen.

[0073] On the front 18, the cooking appliance 2 has a door 19 through which a cooking chamber 16, in or on which the food to be cooked can be placed, can be closed.

[0074] In the slightly top-down perspective illustration according to Fig. 1, a heating element 14 is schematically visible, which is arranged in a base area of ​​the cooking chamber 16 of the cooking appliance 2 and is designed to cook the food or to generate a cooking temperature in the cooking chamber 16. In this case, the heating element 14 is designed to provide bottom heat in the cooking chamber 16 for cooking the food from below. It is understood that the cooking appliance 2 can also have several heating elements 14 in other embodiments. For example, the cooking appliance 2 can alternatively or additionally have a heating element for providing top heat in the cooking chamber 16 for cooking the food from above.

[0075] Furthermore, the cooking appliance 2 can have a convection fan 5 on a rear inner wall of the cooking chamber 16 to provide a convection function for cooking the food.

[0076] The cooking device 2 further comprises an optical sensor 22 for capturing an image of the food being cooked in the cooking chamber 16, the arrangement of one or more optical sensors in the cooking chamber being arbitrary. The cooking device 2 also comprises a control and computing unit 24 and a storage unit 26. The storage unit 26 is integrated into the control and computing unit 24 in this instance, but can also be arranged at a distance from it. In other embodiments, the control and computing unit 24 and / or the storage unit 26 can also be provided outside the cooking device 2, for example in a cloud or on a server. Such an embodiment is also encompassed by the present invention.The control and computing unit 24 is configured to extract information about the food being cooked from the captured image and to store at least the extracted information in a user-specific food history in a database R1 (see Figures 2 and 3) on the storage device 26, wherein the cooking device 2 is configured such that the user-specific food history can be manipulated by a user and / or evaluated by at least one machine learning model, in particular a classification model, in order to optimize automatic food recognition, in particular continuously, and / or to automatically make at least one new operating program suggestion to the user for food that can be placed in or on the cooking device 2.

[0077] In particular, the control and computing unit 24 is also designed to perform at least the following steps: Extracting features of the food being cooked from the captured image, preferably by means of an image feature extraction algorithm;

[0078] Comparing the extracted features with features of at least one standard cooking program food pre-stored in storage device 26, and determining a first comparison result based on this; depending on the first comparison result, comparing the extracted features with features of at least one food item from a user-specific cooking history stored in storage device 26, and determining a second comparison result based on this; operating the cooking device 2 to cook the food depending on the first comparison result or depending on the second comparison result; and storing at least the captured image in a user-specific database of storage device 26 to update the user-specific cooking history.

[0079] Figures 2 and 3 illustrate exemplary embodiments of the present method using schematic flowcharts.

[0080] In step S1, the food to be cooked is preferably placed in or on the cooking appliance 2. In the case of a kitchen appliance 2, as shown by way of example in Fig. 1, the food to be cooked can, for example, be placed in the cooking chamber 16.

[0081] In step S2, an image of the food being cooked is captured by the optical sensor 22, for example a camera.

[0082] In step S3, an image feature extraction algorithm, executable on the control and computing unit 24, extracts features of the food being cooked from the captured image in order to compare them, for example, with features of previously known foods being cooked. The image feature extraction algorithm can be implemented, for example, as an autoencoder, a convolutional neural network (CNN), or a transformer.

[0083] In step S4, the extracted features are compared with features of at least one pre-stored standard cooking program food. Based on this comparison, an initial comparison result is determined and provided. For example, a manufacturer of the kitchen appliance 2 can store several standard cooking program foods in a database of the storage device 26 in order to provide automatic food recognition by the cooking appliance 2, at least for these standard cooking program foods.

[0084] If the comparison from step S4 yields the first comparison result (in Figure 2, "N") that the placed food, particularly with sufficient probability, does not correspond to any standard cooking program food, then, depending on this first comparison result, in step S5 the extracted features are further compared with features of at least one food from a user-specific cooking history, which may also include user-specific cooking programs that are not part of the standard cooking program foods. Based on this, a second comparison result is provided. The user-specific cooking history preferably also includes user-specific foods that the user has previously prepared with the cooking appliance 2, in addition to the standard cooking program foods.

[0085] If the comparison from S5 yields as a second comparison result (in Figure 2 "N") that the placed food, in particular with sufficient probability, does not correspond to any food in the user-specific food history, the user can, in an optional step S6, start a food operating program to cook the food by means of a manual user input, for example via at least one of the controls.

[0086] If the comparison from step S4 yields as the first comparison result (in Figure 2 "J") that the placed food corresponds, in particular with sufficient probability, to a standard cooking program food, a standard operating program or a standard cooking program for cooking the placed food can be suggested to the user in an optional step S7.

[0087] If the comparison from S5 yields a second comparison result (in Figure 2, "J") indicating that the placed food corresponds, with sufficient probability, to a food item in the user's specific cooking history, an optional step S7 can suggest a standard operating program or a standard cooking program to the user for cooking the placed food. Comparisons S4 and S5 can also be combined or executed in parallel.

[0088] In an optional step S8, the user can either accept the program suggestion from S7 (in Figure 2, "J"), so that in a subsequent step S9, the cooking appliance 2 is operated using the suggested standard cooking program or operating program. Alternatively, the user can reject the program suggestion from S7 (in Figure 2, "N"), so that in the optional step S6, the user can start a food-specific operating program to cook the food by manual user input, for example, via at least one of the control elements.

[0089] The image captured of the food being cooked, the features extracted from it, and any further information, such as comparison results, at least one program suggestion, user input, and / or other metadata, are preferably stored in a user-specific database R1 of the storage device 26 in order to update the user-specific food history. It should be noted that standard cooking program foods can also be stored in the user-specific database R1, especially if the user has prepared them with the cooking appliance in the past. Saving can preferably occur following one of steps S6 or S9. However, saving independently of the steps, for example, temporary storage, is also possible.

[0090] Figure 3 shows in detail the possibilities that the user can have for active or passive interaction with the user-specific cooking history stored in the user-specific database R1.

[0091] For example, the user can manipulate the cooking history via manual or voice input using input device 21. Input device 21 is configured to capture user input and provide it to the control and computing device 24.

[0092] Based on user input, the user can manage their personal cooking history in step S10a. This optional step S10a can include several possible sub-steps. For example, naming and / or grouping information stored in the cooking history in step S10a-1. For example, managing and / or deleting information stored in the cooking history in step S10a-2. For example, weighting and / or rating information stored in the cooking history in step S10a-3. The user can also, for example, add further images to a cooking history entry in step S10a-4, showing, for example, different cooking levels.In an optional step S10b, the user can initiate the saving of manually made changes by means of a user input, which leads to the updating of the database R1, which is marked RT in Figure 3.

[0093] According to the invention, adding images is also possible via an app. Likewise, saving can occur automatically in the background without requiring any active intervention from the user.

[0094] The cooking device 2 can also suggest possible updates to the cooking history and, if necessary, further automation options based on the cooking history.

[0095] In step S11a, for example, a (newly) stored image of a food item or its image features in the cooking history can be classified with respect to the foods already stored in the cooking history using a machine learning classification model, which can be executed in particular on the control and computing unit 24. Depending on the classification result, it can then be identified as a food item potentially unknown to the cooking appliance 2 and / or as a potentially unknown food item class (candidate class). The classification of the (newly) stored image with respect to the other images is preferably carried out using a distance metric.The distance metric determines, in a feature space 402 (see Figures 4 and 6), preferably the distance of a feature vector representing the image features of the (new) image to the respective feature vectors of images of cooked goods already available in database R1, or to aggregated feature vectors of a group of images, each representing, for example, a class of cooked goods (e.g., cake, poultry, meat, roast, etc.). In step S11b, the classification or comparison result can also be subjected to a robustness assessment to verify whether a (new) image of a potentially unknown cooked good differs sufficiently from images of cooked goods already present in the cooked goods history.Based on the classification result and / or as a result of the robustness assessment, an operating program for the new food or food class can optionally be suggested to the user in step S11c. Such a classification-based program suggestion can be modified and / or extended by the user in an optional step S11d with user-specific specifications. The information on the new food and / or food class is preferably stored together with the associated operating program so that the database R1 can be updated to RT.

[0096] In an optional step S12, the user can also manually or via voice command add a new cooking program to a new food item. In an optional sub-step S12-1, program specifications can be added and / or defined. Furthermore, in an optional sub-step S12-2, the user can manually or via voice command assign and / or capture at least one image of the associated food item to a new cooking program, and / or select an image from the cooking history.

[0097] In an optional step S13a, the user may have the option to provide a recipe. This can be done by linking to an internet link where the recipe can be accessed and / or by uploading a picture of the recipe.

[0098] In an optional step S13b, (text) features can be extracted from the recipe using an image-to-text extraction algorithm and / or a text extraction algorithm, which can be executed by the control and computing unit 24, on the basis of which, for example, a new operating program for operating the cooking appliance 2 can be determined and / or proposed.

[0099] The recipe and / or the features extracted from it can be linked in an optional step S13c with features and / or the at least one associated image of the food being cooked, which are stored in the user-specific food history, and the food history can be updated based on this.

[0100] In an optional step S14, the user can, for example through an output and / or display device (not shown) of the cooking appliance 2, view statistics and / or other information about the user-specific cooking history, such as how often he has prepared a particular dish.

[0101] In an optional step S15, the user can also directly start a cooking program or operating program for a food item via the input device 21 and / or the preferably combined output and / or display device, for example, without having to compare image features. By adding user-specific images of foods and / or associated cooking programs, or by means of the cooking history, which is preferably continuously updatable, the image feature recognition algorithm can be trained in an optional step S16 so that, during operation of the cooking appliance 2, it can also automatically recognize user-specific foods and the user's stored preparation preferences when a corresponding food item is placed in or on the cooking appliance.

[0102] The user-specific training of the image feature recognition algorithm can be further improved by an optional substep S16-2 of data augmentation or enrichment to generate user-specific cutoff values ​​in the data. Additionally, during training, synthetic data can be generated in an optional substep S16-2, and few-shot learning strategies, or alternatively image generation methods using generative AI, can be applied to overcome data deficiencies in the training data. Furthermore, in an optional substep S16-2, the distance of a cooking item to other cooking items or cooking item classes in the vector space can be modeled to achieve a sharper demarcation.

[0103] Figure 4 shows an exemplary classification of a feature vector 400, which was determined for a piece of food captured as an image by feature extraction, with respect to known food classes K1, K2, K3, K4. Feature extraction projects the information from the images into the high-dimensional feature space 402. A distance D to each food class K1, K2, K3, K4 is calculated, for example, using a distance-based algorithm. The classification takes place in the feature space 402, e.g., with a distance-based classifier such as the K-Nearest Neighbors (KNN) algorithm, where the feature vector 400 is assigned to the food class (in this case, K1) to which the k nearest class members belong. Any distance metric can be used, e.g., the Euclidean or Mahalanobis distance metric.

[0104] Figure 5 shows various cluster representations that may indicate a class distribution of cooking goods. The first column shows individual, convex clusters or groups of cooking goods. The second column shows individual, non-convex clusters. The third column shows unconnected clusters of cooking goods.

[0105] Different approaches can be used to generate cluster-representative feature vectors for the various clusters, as shown in rows 2 and 3 for each cluster type from columns 1-3. A cluster-representative feature vector can be generated by representing a centroid of the cluster (see row 2). A cluster-representative feature vector can also be generated using representatives with subsamples (see row 3). Alternatively, a cluster can also be generated using a diagram- or graph-based representation (see row 4).

[0106] In other words, class distributions can be represented by single feature vectors, such as centroids, or by other, typically subsample-based, sets of feature vectors that serve as class representatives. Class distributions can also be represented by single feature vectors, such as centroids, or by other, typically subsample-based, sets of feature vectors that serve as class representatives, or by graphs that capture spatial relationships between feature vectors. Distribution representations, such as Gaussian distributions (mean plus covariance matrix) or mixture distributions, such as mixtures of Gaussian distributions, etc., can also be used.

[0107] Figure 6 shows a robust and iterative method for adding a new food item or food class to the food history and for automatic food recognition. This addition is preferably performed iteratively. For example, if a customer adds an image of a new food item to the food history, or if the image is captured by the optical sensor 22 and added to the food history, additional image data can be artificially generated for this image (at least one image) through data enrichment or generative classification to train the image feature recognition algorithm. In the feature space representation shown in Figure 6, data enrichment generates additional feature vectors (see the black-bordered white dots) based on the captured or user-provided image stored in database R1.Based on this, potentially new candidate classes for the food included in the captured or provided image can be identified, for example, using unsupervised learning methods such as clustering (see ellipsoids in Figure 6), in order to determine a potentially addable, user-specific new food class nGK. A robustness check can be performed to determine whether the identified class candidate has sufficient distance from the existing standard classes SK1, SK2, and / or the user class(es) BK1, or whether there is no overlap, for example, using a distance-based class or cluster distribution (see lines for adjacent class distributions / clusters). If the robustness check is positive, or if the user overrides the result of the robustness check, a new food class, e.g.,with a user-defined name and / or cooking parameters, added to the (user-defined) classes (BKs) in the database R1.

Claims

Patent claims 1. Cooking device (2) for cooking food, the cooking device comprising an optical sensor (22), a control and computing unit (24) and a storage unit (26), wherein the optical sensor (22) is configured to capture an image of food that can be placed in or on the cooking device (2), and wherein the control and computing unit (24) is configured to perform the following steps: Extracting features of the food being cooked from the captured image, preferably using an image feature extraction algorithm; Comparing the extracted features with features of at least one standard cooking program food pre-stored in the storage device (26), and determining a first comparison result based on this; depending on the first comparison result, comparing the extracted features with features of at least one food from a user-specific cooking history in the storage device (26), and determining a second comparison result based on this; Operating the cooking appliance (2) to cook the food depending on the first comparison result or depending on the second comparison result; and Storing at least the captured image in a user-specific database (R1) of the storage device (26) to update the user-specific cooking history.

2. Cooking appliance (2) according to claim 1, wherein the features of the at least one standard cooking program food are stored by the manufacturer in a standard food database on the storage device (26), and wherein the features of the at least one food in the user-specific food history can be generated by a user, in particular continuously, during operation of the cooking appliance (2) based on the respective captured image and can be stored in the user-specific database (R1) on the storage device (26).

3. Cooking appliance (2) according to claim 1 or 2, wherein the comparison depending on the first comparison result means that the extracted features are compared with the features of the at least one food item in the user-specific cooking history if the first comparison result indicates that the extracted features do not match features of the at least one standard cooking program food item.

4. Cooking appliance (2) according to one of the preceding claims, wherein, - if the second comparison result indicates that the extracted features do not match features of the user-specific cooking history, the operation of the cooking appliance (2) for cooking the food is carried out based on the second comparison result by a user input, in particular manual input, and - if the second comparison result indicates that the extracted features match features of the user-specific cooking history, an operating program proposal, in particular including a user query to accept or reject it, is issued to the user based on the cooking history, and the cooking appliance is operated based on this (2).

5. Cooking device (2) comprising an optical sensor, a control and computing unit (24) and a storage unit (26), wherein the optical sensor (22) is configured to capture an image of food that can be placed in or on the cooking device (2), wherein the control and computing unit (24) is configured to extract information about the food from the captured image and to store at least the extracted information in a user-specific food history in a database (R1) on the storage unit (26), wherein the cooking device (2) is configured such that the user-specific food history can be manipulated by a user and / or evaluated by at least one machine learning model, in particular a classification model, in order to perform automatic food recognition, in particular continuously, based on this.to optimize and / or to automatically provide the user with at least one new operating program suggestion for food that can be placed in or on the cooking appliance (2).

6. Cooking appliance (2) according to one of the preceding claims, wherein the storage of at least the captured image in the user-specific database (R1) comprises: Storing an operating program for operating the cooking appliance (2) for cooking the food; and / or Storing manual user input data that can be generated for operating the cooking appliance (2) depending on the first comparison result and / or depending on the second comparison result; and / or Storing metadata that can be generated depending on the first comparison result and / or depending on the second comparison result for operating the cooking appliance (2).

7. Cooking appliance (2) according to one of the preceding claims, further comprising an input device (21) which is configured to capture user input and provide it to the control and computing device (24), wherein the control and computing device (24) is configured, based on the user input, to: manage and / or supplement the cooking history and / or weight and / or evaluate at least one cooking item in the cooking history, and update changes in the user-specific cooking history; and / or start an operating program included in the cooking history, in particular a user-specific one; and / or create a new operating program for operating the cooking appliance (2), and preferably assign program specifications and / or at least one image of the user-specific cooking history to the new operating program.

8. Cooking device (2) according to one of the preceding claims, wherein the control and computing device (24) is configured to link features and / or the image of the at least one food item stored in the user-specific food item history with a recipe provided by a user for the preparation of the at least one food item and to update the food item history on the basis thereof.

9. Cooking device (2) according to claim 8, wherein the recipe provided by the user comprises a link to an internet link via which the recipe can be accessed, and / or is provided by an image of the recipe, wherein the control and computing device (24) is configured to extract features from the recipe by means of an image-to-text extraction algorithm and / or a text extraction algorithm, on the basis of which a new operating program for operating the cooking device (2) can be created in the user-specific cooking history and on the basis of which the cooking history can be updated.

10. Cooking appliance (2) according to claim 9, wherein the control and computing device (24) is configured to enrich the features and / or the image of the at least one food item that can be linked to the recipe by data augmentation and to update the food item history based thereon in order to improve the automatic food item recognition and the creation of the new operating program, in particular by distinguishing it from stored food items and / or operating programs.

11. Cooking appliance (2) according to one of the preceding claims, wherein the control and computing device (24) is configured to enrich the information by which the cooking history can be updated, at least partially, by data augmentation or generative AI, and to update the cooking history based thereon.

12. Cooking appliance (2) according to one of the preceding claims, wherein the control and computing device (24) is configured to determine, by means of a classification model of machine learning, on the basis of information by which the cooking history can be updated, a cooking product that is potentially unknown to the cooking appliance (2) and / or a potentially new operating program for the cooking appliance (2).

13. Cooking device (2) according to claim 12, wherein the machine learning classification model has a distance metric by means of which the information by which the cooking history can be updated is comparable in a classification space with information stored in the cooking history and / or the storage device (26).

14. Cooking appliance (2) according to claim 13, wherein, if the information by which the cooking history can be updated with respect to the information stored in the cooking history and / or the storage device (26) meets a classification criterion, the control and computing device (24) is configured to suggest to a user the cooking food previously unknown for the cooking appliance (2) and / or the new operating program for the cooking appliance (2), wherein preferably the suggested cooking food and / or the suggested operating program can be modified and / or extended by the user with user-specific specifications.

15. Method for operating a cooking appliance (2) for cooking food and for updating a user-specific food history, the method comprising the steps: Placing (S1) the food in or on the cooking appliance (2); Capturing (S2) an image of the food being cooked by an optical sensor; Extracting (S3) features of the food being cooked from the captured image, in particular by means of an image feature extraction algorithm; Comparing (S4) the extracted features with features of at least one pre-stored standard cooking program food, and based on this determining an initial comparison result; Depending on the first comparison result, compare (S5) the extracted features with features of at least one cooking item from a user-specific cooking history and determine a second comparison result based on this; Operating (S6 / S9) the cooking appliance to cook the food depending on the first comparison result or depending on the second comparison result; and Saving at least the captured image to a database (R1) to update the user-specific cooking history.

Images