Identifying cooking item in

Abstract

The invention relates to a method (S1-S9) for identifying an item (G) to be cooked in at least one image point-based thermal image (W1, W2) recorded from a cooking chamber (2) of a microwave cooking appliance (1), in which microwave energy is radiated into the cooking chamber (2), the thermal image (W2) is recorded, the following image points are identified, the image points have a temperature value that is increased by a preset temperature level with respect to a set of all image points, and executing a filling algorithm that classifies image points belonging to an image surface bordered by at least some of the identified image points as the item of cooking (G). The invention further relates to a microwave cooking appliance (1) having a cooking chamber (2) to which microwaves can be applied, a thermal image camera (11) provided for recording thermal images (W1, W2) from the cooking chamber (2), and a control device (13), the microwave cooking appliance (1) being configured to carry out the method (S1-S9; s1 to S10). The invention can be advantageously applied in particular to domestic microwave cooking appliances.

Description

Technical Field

[0001] The invention relates to a method for identifying a cooking item in at least one pixel-based thermal image recorded from a cooking chamber of a microwave cooking device, wherein microwave energy is radiated into the cooking chamber, a thermal image is recorded, and pixel values ​​are identified which have a temperature value which is increased by a predetermined temperature level relative to the set of all pixel values. The invention also relates to a method for operating a microwave cooking device, wherein the position of the cooking item in the thermal image is determined according to the method, and the cooking process is controlled according to an evaluation of the pixel values ​​attributed to the cooking item. Furthermore, the invention relates to a microwave cooking device having a cooking chamber which can be loaded with microwaves, a thermal image camera which is provided for recording a thermal image from the cooking chamber, and a control device, wherein the microwave cooking device is configured to carry out the method. The invention can be applied in particular advantageously to domestic microwave cooking devices. Background Art

[0002] For an in-depth control of the cooking process in a microwave cooking appliance in order to improve the cooking result, which control is based on image-based infrared measurements (IR) or thermal radiation measurements, it is important to accurately determine the cooked item. This means, in particular, that an accurate distinction should be made to determine whether an image point imaged by the field of view of the thermal imaging camera is associated with or belongs to a sub-region of the cooked item or - irrelevant for the control of the cooking process - is associated with the environment of the cooked item, such as a cooked item carrier (e.g. a plate, grill, baking tray, etc.) or a cooking chamber wall.

[0003] The identification of areas associated with cooking in the image of the cooking chamber can be done by the user by manual marking on a touch screen ("Tocuhscreen"). However, this is not user-friendly and is often inaccurate. In addition, complex image evaluation methods can be used, in particular methods based on artificial intelligence ("AI"), in order to automatically identify cooking areas in the image. However, such methods require a lot of calculations and training.

[0004] WO 2020 / 156928 A1 discloses a household cooking device. The household cooking device comprises a cooking chamber heating device, which is configured to locally heat a cooking chamber and can be operated in at least two configurations, which generate different energy distributions in the cooking chamber; a temperature detection device, which is configured to non-contactly detect the heat distribution in the cooking chamber; a data processing device, which is configured to distinguish a non-cooking area of ​​the cooking chamber from at least one area occupied by cooking objects from the detected heat distribution; and a control device, which is configured to set the current configuration of the cooking chamber heating device in view of the increase in energy output entering the identified cooking object and control the cooking chamber heating device, the temperature detection device and the data processing device, wherein the data processing device is configured to distinguish the non-cooking area from the cooking object in the detected heat distribution according to the temperature difference between the non-cooking area and the cooking object.

[0005] WO 2020 / 156929 A1 discloses a microwave device. The microwave device has a microwave device configured to generate microwaves and introduce the microwaves into a cooking chamber and the microwave device can be operated in at least two configurations, which generate different field distributions of microwaves in the cooking chamber; a temperature detection device configured to detect the heat distribution in the cooking chamber in a non-contact manner; a data processing device configured to identify a non-cooking area in the cooking chamber from the detected heat distribution; and a control device configured to set the current configuration of the microwave device and operate the microwave device, wherein the control device is configured to select or set at least one configuration of the microwave device in view of the reduction of microwave power in the identified non-cooking area.

[0006] EP 3767580 A1 discloses a control unit for a household appliance, the household appliance having at least one reference marker in an interior space, wherein the reference marker has reference values ​​for one or more different characteristics. The control unit is configured to detect image data related to the interior space of the household appliance by means of a camera of the household appliance and to recognize the reference marker in the image data. In addition, the control unit is configured to determine actual values ​​of one or more characteristics of the reference marker based on the image data. The control unit is also configured to process the image data according to the actual values ​​and according to the reference values ​​to determine object information related to an object in the interior space of the household appliance and / or to provide an evaluated image related to the interior space of the household appliance.

[0007] DE 102017101183 A1 discloses a method for operating a cooking device and a cooking device, wherein a cooking item in a cooking chamber is heated by a heating device. The cooking item is detected by a camera device. Based on the detection of the cooking item, at least one cooking item characteristic is determined. In this case, the heating device comprises a heating source having a plurality of individually controllable heating mechanisms. At least one heating mechanism is used to heat one of a plurality of spatial segments in the cooking chamber in a targeted manner. The control of each heating mechanism is performed according to the cooking item characteristic.

[0008] WO 2016170734 A1 discloses a cooking device, comprising: a microwave generating unit generating microwaves; a heating chamber for accommodating an object to be heated; an infrared sensor installed in the heating chamber; a scanning unit for moving the infrared sensor for scanning; and a control unit for controlling the microwave generating unit based on the output of the infrared sensor. The infrared sensor acquires a large number of temperature distributions by acquiring one temperature distribution each time the infrared sensor is scanned over a predetermined distance. The control unit controls the microwave generating unit according to the temperature distribution acquired by summarizing the large number of temperature distributions.

[0009] US10219330 B2 discloses an electronic oven and an accompanying control system that avoids boiling or splattering in a chamber of the oven during heating of an object in the heating chamber. A method that can be performed by the control system includes evaluating sensor data from a visible light sensor and sensor data from an infrared light sensor. A controller is communicatively coupled to the visible light sensor and the infrared light sensor. The method is intended to generate a splattering prediction in response to the evaluation of the sensor data of the visible light sensor and the sensor data of the infrared light sensor. The method is also intended to reduce the power level of a microwave energy source in response to the splattering prediction. The controller is also communicatively coupled to the microwave energy source.

[0010] EP 2618634 A1 discloses a microwave heating device and a method for heating a load using microwaves. The microwave heating device comprises a cavity arranged to accommodate the load, a plurality of delivery openings for delivering microwaves from a plurality of microwave generators to the cavity, and a control unit. The control unit is configured to obtain a desired temperature pattern in the cavity based on information about a plurality of regions of the load, determine a heating pattern including corresponding regions of different intensities, set to the desired temperature pattern, and control at least some of the plurality of microwave generators to provide the heating pattern in the cavity.

[0011] WO 2012 / 109634 A1 discloses a device for treating an object using HF energy. The device may include a display for displaying an image of the object to be treated to a user, wherein the image includes at least a first segment and a second segment of the object. The device may also include an input unit and at least one processor, wherein the processor is configured to: receive information based on an input provided to the input unit; and generate processing information used in treating the object based on the received information to achieve a first processing result of a first portion of the object and a second processing result of a second portion of the object.

[0012] EP 1997349 B1 discloses an electromagnetic heating device for heating an irregularly shaped object, comprising: a cavity for placing the object; at least one delivery portion for feeding UHF energy or microwave energy into the cavity; and a control unit, which controls one or more characteristics of the cavity or the energy to ensure that the UHF energy or microwave energy is uniformly distributed over at least 80% of the volume of the object within ±30%.

[0013] WO 2020 / 200913 A1 discloses a household appliance. The household appliance comprises a processing chamber for processing items (especially cooking items); at least one pattern light configured to irradiate at least one light pattern into the processing chamber; and at least one image sensor pointing to the processing chamber for recording at least one light pattern reflected from the processing chamber, wherein the pattern light can be rotated by means of a motor, and the household appliance is configured to determine at least one contour information of an item illuminated by the light pattern based on at least two reflected light patterns associated with different rotation angles of at least one pattern light. The method is used to determine the contour information of an item located in the processing chamber of the household appliance. WO 2020 / 200913A1 can be particularly advantageously applied to determine the contour information of cooking items in an oven.

[0014] EP 3574711 A1 discloses methods and systems related to an improved human-machine interface for an electronic oven. A plurality of methods for displaying information to a user are disclosed. A plurality of methods for distributing segmentation and recognition tasks between a user and a control system are disclosed. In one example, an electronic oven includes a touch display screen, a heating chamber for heating an object, a light sensor having a field of view of at least a portion of the heating chamber, and a microwave energy source coupled to the heating chamber. The oven also includes a computer-readable medium storing instructions for displaying a segment of the heating chamber as an image on the touch display screen using information from the light sensor, and processing touch input on the image.

[0015] EP 3767183 A1 discloses a method for identifying dirt on a household appliance. The method comprises the steps of providing a reference image of a cavity or a segment of the household appliance, detecting a current image of the same cavity or segment, comparing the current image with the reference image and generating a differential image of the same cavity or segment, and checking whether pixels and / or groups of adjacent pixels of the differential image exceed a predefined threshold value.

[0016] EP 2 055 146 B1 discloses how to determine the geometry of an object by measuring with HF energy.

[0017] DE 102020215681 A1 discloses a household microwave appliance, which is operated in sequence with multiple parameter configurations, which process cooking objects in different local ways, so as to perform an initial scan with the help of a thermal image sensor directed to a cooking chamber for determining the temperature distribution on the surface of the cooking objects, so as to obtain a change pattern consisting of differences in different temperature distributions, calculate an evaluation value for the change pattern, and determine the following heating mode based on a target temperature distribution (which is obtained from a normalized target state and a current temperature distribution), wherein the heating mode makes the current temperature distribution optimally close to the target temperature distribution, and then load the cooking object with microwave power with the parameter configuration associated with the heating mode. Summary of the invention

[0018] The object of the present invention is to at least partially overcome the disadvantages of the prior art and in particular to provide an improved possibility for identifying cooked items in pixel-based images recorded from a cooking chamber of a microwave cooking appliance.

[0019] This object is achieved according to the features of the independent claim. Preferred embodiments can be gathered in particular from the dependent claims.

[0020] The object is achieved by a method for identifying cooked food in at least one pixel-based thermal image recorded from a processing chamber ("cooking chamber") of a microwave cooking device, wherein:

[0021] - radiates microwave energy into the cooking chamber,

[0022] - record thermal images,

[0023] - identifying image points which have a temperature value which is increased by a predetermined temperature level relative to the set of all image points,

[0024] - executing a filling algorithm which assigns image points belonging to an image area enclosed by at least some of the identified image points to culinary products.

[0025] This method has the following advantages: it is very precise, user-friendly, computationally inexpensive and therefore very fast, and can be run automatically, for example compared to manual marking on a touch screen or automated marking by KI image recognition methods.

[0026] The method makes use of the so-called "edge overheating effect", in which the heating image shows a frequently occurring behavior: the edge of the cooked product is strongly heated at several locations when exposed to microwave radiation (especially initially), while the inner areas of the surface of the cooked product are usually heated much less or even insignificantly. This "edge overheating effect" occurs in practically all cooked products with a water content. Although this effect in principle makes the cooking process more difficult, since an increased energy input is found at the edge, while the center is difficult to heat, this phenomenon now provides a computationally less complex way to detect the outer contour of the cooked product and thus distinguish between image points belonging to the cooked product and those that do not. This relationship can also be referred to as "surface masking".

[0027] Recognizing the cooking item in the thermal image corresponds in particular to associating an image point with whether a temperature is detected on the surface of the cooking item or, conversely, whether the temperature is not detected on the surface of the cooking item but, for example, on the surface of another object, such as a cooking item support, a cooking chamber wall, etc. Recognition therefore corresponds in particular to recognition of the position of the cooking item and not to recognition of the type of cooking item.

[0028] The cooking chamber is used to accommodate the processed items to be processed by microwaves, especially the processed items to be cooked. The cooking chamber can be loaded with microwaves. The cooking chamber usually has a loading opening on the front side, which can be closed in a microwave-tight manner by means of a cooking chamber door.

[0029] The microwaves are generated by means of at least one microwave generator, for example a magnetron or a semiconductor-based microwave generator. The microwaves can be introduced from the microwave generator directly or via a corresponding microwave guide device into the cooking chamber. One extension is that the microwaves are introduced into the cooking chamber via a microwave feed point ("microwave port") or via a plurality of microwave ports, in the case of a plurality of microwave ports, if necessary with a phase offset (in particular a variably settable phase offset). There can also be a rotatable rotating antenna and / or a stirrer present on the microwave port to change the pattern image of the microwaves in the cooking chamber. In addition, the pattern image in the area of ​​the cooking object can also be changed by setting a turntable (if present). The microwave frequency can be in particular in the range between 2.4 GHz and 2.5 GHz, in particular about 2.45 GHz, or in the range between 902 MHz and 928 MHz, in particular about 915 MHz. In particular, a semiconductor-based microwave generator can change the microwave frequency in a targeted manner.

[0030] In particular, a pixel-based thermal image is constructed from a matrix-like arrangement of pixels or pixels, wherein the value associated with a pixel corresponds to the temperature in the surface region measured by the respective pixel.

[0031] The image point-based thermal image is typically recorded by a digital thermal image camera. For large-area detection of the cooked food, the thermal image camera is advantageously arranged in the region of the top of the cooking chamber wall and is particularly oriented downward or diagonally downward. The field of view of the thermal image camera advantageously includes at least the bottom of the cooking chamber wall and, if necessary, also parts of the sides of the cooking chamber wall.

[0032] However, the microwave cooking appliance can also have a plurality of thermal imaging cameras which advantageously record images of the cooking chamber from different angles.

[0033] In one embodiment, the microwave cooking device has, in addition to at least one digital thermal image camera, at least one digital "optical" cooking chamber camera that records images in the spectral range. This has the advantage that the cooking process can be controlled based on the optical properties of the surface of the cooked food (such as browning, color change, volume change, etc.). The images recorded by it can also be displayed to the user. The optical cooking chamber camera can be, for example, a digital RGB color camera. An advantageous embodiment for detecting the cooked food over a large area is that the optical cooking chamber camera is arranged in the area of ​​the top of the cooking chamber wall and is especially oriented downward or obliquely downward. In one embodiment, the thermal image camera and the optical cooking chamber camera are arranged close to each other, which has the advantage that the thermal image and the optical image show particularly similar areas of the cooking chamber and are particularly easy to compare.

[0034] By irradiating microwave energy into the cooking chamber, the food contained therein is heated, with the edges of the food being heated not only more strongly than the "non-food" surroundings outside it, but also, due to the "edge overheating effect", usually more strongly than the center of the food. After a certain irradiation duration, a thermal image of the cooking chamber is recorded, in which the temperature distribution of the surface of the food is typically imaged.

[0035] All image points of the thermal image have corresponding temperature values, wherein at least some image points associated with the edge of the cooking object can be identified in the following manner: they have a significantly higher temperature level or display a significantly higher temperature than other image points. For example, image points with a higher temperature level can be distinguished from other image points by threshold comparison, thereby identifying image points with a higher temperature level. These image points with a higher temperature level at least display a fragment of the edge or outer contour of the cooking object from the perspective of the thermal image camera.

[0036] The filling algorithm then fills the surface bounded by the image points associated with the edge of the cooking object in the image plane, or identifies the image points located in such a surface. After the filling algorithm has been executed, the image points located in the surface (including its edge) are classified as cooking objects. Conversely, the image points located outside the surface (including its edge) are not classified as cooking objects.

[0037] In one embodiment, the filling algorithm is designed such that after execution of the filling algorithm, image points previously associated with edges of the cooking product are no longer associated with surfaces corresponding to the cooking product, but with non-cooking product areas, because they do not meet certain criteria of the filling algorithm for association with surfaces.

[0038] One design is to record a first thermal image before radiating microwave energy into the cooking chamber; record a second thermal image after radiating microwave energy into the cooking chamber; create a differential thermal image, wherein the temperature difference between the temperature value recorded by the second thermal image and the temperature value recorded by the first thermal image is calculated for each image point and the image point in the differential thermal image is identified. This has the following advantages: after placing the food in the cooking chamber, the temperature difference between the food and the environment of the food or within the food is taken into account and the influence of the temperature difference is neutralized.

[0039] One design solution is to map or "normalize" the temperature difference to a value range [0; 1] (or another preset value range). This results in the advantage that the method can be more easily adapted to different types of cooked food and / or to different microwave operating parameters during microwave feed.

[0040] One embodiment is to identify those image points that reach or exceed a predefined temperature threshold value as being associated with the edge of the cooking product. This is advantageously particularly easy to implement.

[0041] One embodiment provides that the temperature threshold lies in the range [0.2; 0.5] when the temperature difference is mapped into the value range [0; 1]. This has proven to be a particularly good compromise between exact detection of the edge of the cooking product and a sufficiently closed edge. In the case of too low a value of the temperature threshold (e.g. 0.1), although the cooking product G can usually be completely detected, large areas of the cooking product support or the cooking chamber wall may also be erroneously identified as belonging to the cooking product. If the temperature threshold is selected too high (e.g. 0.7), only individual particularly strongly heated edge segments of the cooking product G are detected, which may not be sufficient to reliably define the edge filled by the Starfill algorithm. In addition, it is advantageous that the same threshold can be applied to many different cooking products and / or different microwave operating parameters by mapping the value range into a fixed range.

[0042] One embodiment is to use the following algorithm as the filling algorithm (hereinafter referred to as the "Starfill" algorithm without general restriction), in which an image point is assigned to a cooked product if the number of directions in which the image point is identified, starting from an image point in a star shape, reaches or exceeds a minimum number. In particular, in contrast to the so-called "Floodfill" algorithm, which is often used in graphics programs to fill a bordered surface with color, this algorithm can avoid gaps in the identified edge leading to an incorrect association of areas outside the edge with cooked products. In addition, the "Floodfill" algorithm requires one of the inner points of the bordered surface as a starting point, which surface is still unknown according to the invention at the beginning of the evaluation.

[0043] In the Starfill algorithm, for a considered image point, it is checked whether there are image points associated with a culinary object in different directions starting from this image point in a star shape in the image plane of the thermal image, in particular the differential thermal image. If a preset minimum number of directions meets this condition, the considered image point is associated with the culinary object. The Starfill algorithm can perform this check and subsequent association for all image points of the thermal image, in particular the differential thermal image, for example row by row and column by column.

[0044] In the case of an (x,y) matrix-shaped thermal image, directions which "star out" from the image points are understood to be directions of symmetry along the x and y extensions as well as any oblique directions lying uniformly between these (main) extensions.

[0045] The minimum number of star directions for an interior point within the (x,y) matrix is ​​four, starting from the image point:

[0046] - in the x direction,

[0047] - in the -x direction,

[0048] - in the y direction, and

[0049] - In the -y direction.

[0050] The next largest number of star directions is eight, i.e., starting from this image point, there are four additional directions through the nearest neighboring image point that are tilted with respect to the x-extension or y-extension. This can in principle be extended as desired to 12, 16, 20, etc. directions. The greater the number of directions, the greater the accuracy of the association at the tilted edges of the edge.

[0051] For edge points and corner points, a reduced number of possible directions are available, i.e., two directions for corner points and three directions for edge points if four directions are usually available for interior points, three directions for corner points and five directions for edge points if eight directions are usually available for interior points, and so on.

[0052] One design is to set at least one of the following boundary conditions for the Starfill algorithm for a total of eight star directions: for corner points, the minimum number is two; for edge points, the minimum number is three; for interior points, the minimum number is six. This has proven to be a particularly good compromise between completely filling the volume bounded by the edge and avoiding associating image points outside the edge with the cooked product. With a total of eight star directions, a maximum of three directions can come from a corner point, a maximum of five directions can come from an edge point, and so on.

[0053] One development is to apply the Starfill algorithm to a specific pixel and then to the next pixel including the results for all previous pixels. The Starfill algorithm can be executed in particular row-by-row and column-by-column.

[0054] One development is that an already recognized pixel is recognized unchanged, ie the association once made with the cooked product is not cancelled, for example even if a predetermined minimum number of directions is no longer given to the pixel or the conditions of the Starfill algorithm are no longer met.

[0055] Another embodiment provides that an image point which has been identified or associated with a culinary item is no longer associated with the culinary item (ie is de-associated) if a predefined minimum number of directions is no longer met.

[0056] In one embodiment, the filling algorithm is executed multiple times ("recursively") for all image points until the identified image points no longer change. The method can thus perform the association of image points iteratively. This is particularly advantageous for associating image points with or not associating them with culinary items as completely as possible.

[0057] Alternatively or additionally, the filling algorithm may be executed multiple times until a maximum number of passes or iterations is reached.

[0058] In one embodiment, the image points associated with the cooking item are additionally converted into image points of an optical digital camera of the microwave cooking device. This is particularly advantageous for improving the optical monitoring of the cooking item (in the visible spectrum) by adapting the method, for example with regard to the degree of browning, volume changes, etc., since information about the position of the cooking item can be used for the optical monitoring. In this case, it is used that the image points of the thermal image camera can be associated with the image points of the optical digital camera, so that when the image points of the thermal image camera are associated with the cooking item, one or more image points of the optical digital camera showing the same area of ​​the cooking item can also be associated with the cooking item. In one embodiment, the image points of the thermal image camera that are not associated with the cooking item are converted into image points of the optical digital camera of the microwave cooking device.

[0059] In one embodiment, this can also be done in reverse, i.e., image points of the optical digital camera that are assigned to or not assigned to a culinary object are converted into corresponding image points of the thermal imaging camera. This means that if at least one image point of the optical digital camera is not assigned to a culinary object, image points of the optical digital camera that show the same area of ​​the culinary object are also not assigned to the culinary object.

[0060] One design solution is to perform the identification of the position of the cooking object at the initial stage of the cooking process. This is particularly advantageous because the cooking object has not yet been fully heated and the "edge overheating effect" is particularly obvious. This in turn improves the reliability of the method.

[0061] One design solution is that during the radiation of microwave energy into the cooking chamber for the execution of the method, at least one microwave operating parameter that changes the pattern of microwaves in the microwave chamber is changed. This advantageously avoids narrow local spatial regions with high microwave energy (i.e., "hot spots"), or at least changes them so much that a larger area of ​​the cooked item is heated. This in turn has the advantage that the increased heating of the edges of the cooked item caused by the "edge overheating effect" can be particularly clearly identified by means of a thermal imaging camera, in particular over the entire edge as far as possible, or the number and / or length of edge segments that are not particularly heated are kept small. This design solution can be particularly advantageously implemented in conjunction with DE 102020215681 A1, in particular within the scope of the "initial scan" mentioned in DE 102020215681 A1, wherein microwaves are fed into the cooking chamber under different parameter configurations, the temperature distribution associated with the parameter configurations at the surface of the cooked item is measured by means of a thermal imaging camera, and the heating pattern is determined based on the differences in the different temperature distributions. These parameter configurations can then correspond in particular to the above-mentioned microwave operating parameters.

[0062] In one extended solution, the microwave operating parameters include at least one of the following parameters:

[0063] - in the case of a rotating antenna: the rotation angle of the rotating antenna, for example in the range of [0°; 180°] or [0°; 360°], for example in steps of 1°, 5° or 10°;

[0064] - in the case of a mode stirrer or agitator: the angle of rotation of the mode stirrer, for example in the range [0°; 180°] or [0°; 360°], for example in steps of 1°, 5° or 10°;

[0065] - in the case of a microwave generator with a variable microwave frequency, in particular a semiconductor-based microwave generator: a microwave frequency, for example in the range [2.4 GHz; 2.5 GHz], for example in steps of 10 MHz;

[0066] - In case there are multiple microwave ports, the phase difference between the microwaves radiated from these microwave ports may vary: the phase difference is, for example, in the range of [0°, 360°].

[0067] Advantageously, the turntable (if present) rotates at least once; the rotating antenna (if present) rotates at least once; the microwave frequency is changed as completely as possible once (if possible); the phase difference is traversed once (if possible), etc. When a turntable is used, the second thermal image is preferably recorded after an integer number of complete revolutions, so that the cooked food is in the same position when the first and second thermal images are recorded. Otherwise, the thermal images must be aligned with each other by means of a suitable rotation matrix, which is also feasible in principle.

[0068] The object is also achieved by a method for operating a microwave cooking device, wherein the position of the cooking object is determined according to the method described above and the cooking process is controlled as a function of the evaluation of at least the image points that are assigned to the cooking object, and optionally also as a function of a combination of the evaluation of the image points that are assigned to the cooking object and the evaluation of the image points that are not assigned to the cooking object. The method for operating a microwave cooking device can be designed similarly to the method described above for identifying the cooking object, and vice versa, and has the same advantages. For example, determining the position of the cooking object can be used to more accurately identify a target cooking state, thereby increasing the success rate and improving the cooking result. When the target cooking state is reached, at least one action can be triggered, such as ending the feeding of microwave radiation and / or outputting a prompt to a user.

[0069] The object is also achieved by a microwave cooking device, comprising a cooking chamber that can be loaded with microwaves, a thermal image camera configured to record thermal images in the cooking chamber, and a control device, wherein the microwave cooking device, in particular the control device thereof, is configured to perform one or more of the above-mentioned methods. The microwave cooking device can be constructed similarly to the method, or vice versa, and has the same advantages.

[0070] The microwave cooking device is in particular a household device. The microwave cooking device can be an independent microwave cooking device or a combined device, such as a microwave cooking device with an additional heat radiator (such as a resistive heating element), in particular in the form of a tabletop device, or an oven with microwave functionality.

[0071] In particular, the microwave cooking device may comprise one or more of the following means:

[0072] a microwave-tight cooking chamber door for microwave-tightly closing a loading opening at the front side of the cooking chamber;

[0073] - at least one microwave generator;

[0074] - at least one microwave feed point;

[0075] - Rotate the antenna;

[0076] - Mode stirrer;

[0077] -Turntable. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] The above characteristics, features and advantages of the present invention and the manner of achieving the same will be more clearly understood and appreciated in the following detailed description of the embodiments with reference to the accompanying drawings.

[0079] Figure 1 A sketch of a microwave cooking device is shown as a sectional view in a side view, which is configured to carry out a method for recognizing a cooking object in an image point-based thermal image recorded from a cooking chamber thereof;

[0080] Figure 2 A possible flow of the method is shown;

[0081] Figure 3 An image point-based thermal image of a cooking item placed on a plate before microwaves are fed in is shown;

[0082] Figure 4 After microwave feeding Figure 2 a point-based thermal image of a cooked item placed on a plate;

[0083] Figure 5 Shows from Figure 3 and Figure 4 The temperature difference image generated from the thermal image;

[0084] Figure 6 shows the association of image points derived from the temperature difference image with the cooked object before applying the Starfill algorithm;

[0085] Fig. 7A shows the application of the Starfill algorithm to the interior points of the temperature difference image;

[0086] Figure 7B The application of the Starfill algorithm to the edge points of the temperature difference image is shown;

[0087] Figure 7C The application of the Starfill algorithm to the corner points of the temperature difference image is shown;

[0088] Figure 8 shows the association of image points derived from the temperature difference image with the cooked object after the first application of the Starfill algorithm;

[0089] Fig. 9The association of the image points derived from the temperature difference image with the cooked product is shown after further application of the Starfill algorithm. DETAILED DESCRIPTION

[0090] Figure 1 A schematic diagram of a microwave cooking device 1 with a cooking chamber 2 is shown as a sectional view in a side view, which is configured to operate for recording pixel-based thermal images W1, W2 (see FIG. 1 ) from the cooking chamber 2. Figure 3 and Figure 4 ) in the cooking chamber 2. The cooking chamber 2 can be closed at its front loading opening by a cooking chamber door 3. In the present case, the microwave cooking device 1 is designed purely by way of example as an oven with microwave functionality, wherein the cooking G can typically be placed on a cooking support 4 located in a specific insert layer, such as a baking tray or a grill, for example, on a cooking container such as a plate 5.

[0091] The microwave cooking device 1 comprises a (especially semiconductor-based) microwave generator 6 for generating microwaves, especially in the range of [2.4 GHz; 2.5 GHz]. The generated microwaves are guided by a microwave guide 7 (especially in the form of a waveguide) to a microwave feed point 8 and fed there into the cooking chamber 2. At the microwave feed point 8, there is a rotatable rotating antenna 9 for changing the mode.

[0092] Furthermore, a pixel-based thermal imaging camera 11 is located at the top 10 of the cooking chamber 2 for recording thermal images W1, W2, which is directed from above toward the cooking chamber 2 and includes the cooking chamber 2 with the cooking G therein in its field of vision. Optionally, the microwave cooking device 1 can also include an optical digital camera 12, which is also arranged in particular at the top 10 of the cooking chamber 2, in particular close to the thermal imaging camera 11.

[0093] The microwave generator 6, the motor for rotating the antenna 9, as well as the thermal imaging camera 11 and, if applicable, the optical digital camera 12 are controlled by means of a control device 13. The control device 13 can also be configured to carry out the method described below.

[0094] Figure 2 A possible sequence of a method for identifying the position of a cooking item G in the thermal images W1 , W2 is shown.

[0095] In step S1 , the food G is placed on the food support 4 and, if necessary, on or in tableware (such as a plate 5 , a bowl, etc.). In addition, the microwave power is set.

[0096] In step S2, a first thermal image W1 of the unheated cooking object G is recorded from the cooking chamber 2. Figure 31 is shown with a temperature scale at the right edge. In the thermal image W1, for example, a plate is shown with a cooked item G at approximately room temperature (here: ravioli and sauce in a round soup plate). The thermal image records IR image points arranged in the form of an (x, y) matrix, whose values ​​correspond to the local surface temperature of the object in the field of view of the thermal image camera 11 or are at least related to the surface temperature. The temperature distribution in the cooking chamber 2 is therefore mapped onto a matrix of (x·y) IR image points, wherein a thermal image camera 11 suitable for use can have hundreds to thousands of individual image points. The resolution of the thermal image W1 shown is (24·32)=768 image points. The method can also be applied to cooked items that are spatially distributed and separated from one another, such as individual potatoes with adjacent pieces of meat.

[0097] In step S3, microwaves with a set microwave power are radiated into the cooking chamber 2, wherein at least one microwave operating parameter that changes the pattern image of the microwaves in the cooking chamber 2 is changed during the radiating, such as the rotation angle of the rotating antenna 9 and / or the microwave frequency. For example, during this initial stage or "initial scanning", the rotating antenna 9 can be continuously rotated. In this case, it is advantageous to traverse as many different field distributions or pattern images as possible in order to cause a temperature increase in as many areas of the cooking object G as possible.

[0098] Step S3 can be performed for a preset duration, for example between 10s and 30s, but the duration can (for example similar to the initial scan described in DE 102020215681 A1) depend, for example, on the set power, the thermal mass of the cooking object G and the absorption capacity of the cooking object G. Advantageously, in step S3, the rotating antenna 9 rotates at least one revolution, the microwave frequency is changed as completely as possible once (if possible), the phase difference is traversed once (if possible), etc.

[0099] After step S3, a second thermal image W2 is recorded in step S4 and drawn on Figure 4 This second thermal image shows that the food G is heated significantly more strongly than the cooking chamber wall or the visible part of the plate on which the food G is located. In particular, the "edge overheating effect" can also be seen, in which the edge of the food G is heated particularly strongly, while the inner area of ​​the surface of the food G is usually heated significantly less, or even hardly at all.

[0100] In step S5, a differential thermal image DW is created (see Figure 5 ), wherein the temperature difference (“temperature rise and fall amplitude”) between the temperature value recorded using the second thermal image W2 and the temperature value recorded using the first thermal image W1 is calculated for each of the image points.

[0101] In step S6, the following image points are identified in the differential thermal image DW, which have a temperature value that is increased by a predetermined temperature level relative to the set of all image points in the differential thermal image DW. These image points form a corresponding image pattern BM0 in the (x, y) matrix M, as shown in Figure 6 as shown in .

[0102] Step S6 can be implemented, for example, as follows: In a first substep S6A, the temperature differences are mapped to a value range [0; 1]. Here, the minimum value of the temperature differences of all 768 image points is set to be equal to the value 0, and the maximum value of the temperature differences of all image points is set to be equal to the value 1. For example, if the minimum value is 3°C and the maximum value is 13°C, the set of temperature differences {3; 5; 10; 13} will be mapped to the set {0; 0.2; 0.7; 1}.

[0103] In the subsequent sub-step S6B, the image points that reach or exceed a preset temperature threshold (for example, 0.27 here) are identified as belonging to the edge of the cooking object G. These image points form the Figure 6 The initial image pattern BM0 before executing the filling algorithm is shown in FIG.

[0104] exist Figure 6 The image points identified in FIG. 1 as belonging to the cooking product G already very clearly show the (approximately circular here) edge or outer contour of the cooking product G. The inner area of ​​the cooking product G has not yet been filled, and there are still openings in the edge at locations that remain relatively cold. This cannot be resolved by generally applicable temperature thresholds.

[0105] Therefore, in step S7 , a filling algorithm in the form of a Starfill algorithm is applied to the initial image pattern BM0, wherein an image point is assigned to the cooked product G if the number of directions starting from the image point star of the cooked product G in which the image point is identified reaches or exceeds a minimum number. In the following, a Starfill algorithm with a total of eight star directions is considered by way of example.

[0106] This is Fig. 7A In the matrix M, for the pixel represented by the solid circle as a corner point, it is indicated that the corner point has three possible directions starting from the corner point in a star shape, namely the x direction, the y direction and the (x, y) oblique direction. Here, by way of example, the pixel (filled) recognized as belonging to the cooked product is located in the x direction and the y direction, as indicated by the solid arrows. In this exemplary embodiment, the distance of the recognized pixel from the corner point is insignificant, but can usually be taken into account. In the (x, y) oblique direction, however, no pixel is recognized, as indicated by the dashed arrow. If, for a corner point, at least one recognized pixel is present in each of at least two of these directions, the corner point is associated with the cooked product G, this would be the case for the corner point shown.

[0107] exist Figure 7B , the Starfill algorithm is shown for the image points represented by solid circles as edge points in the matrix M: the edge points have five possible directions from the edge points in a star shape, namely the x direction, the (-x) direction, the y direction, the (x, y) oblique direction and the (-x, y) oblique direction. In this example, the image points identified as belonging to the cooked product are located in the (-x) direction, the y direction and the (x, y) oblique direction, as indicated by the solid arrows. In contrast, no image points are identified in the x direction and the (-x, y) oblique direction, as indicated by the dashed arrows. This is the case for the edge points shown if, for an edge point, at least one identified image point is associated with the cooked product G in at least three of these directions.

[0108] exist Figure 7C , the Starfill algorithm is shown for an image point represented by a solid circle as an internal point in the matrix M: the internal point has all eight possible directions from the internal point, namely the x direction, (-x) direction, y direction, (-y) direction, (x, y) oblique direction, (-x, y) oblique direction, (x, -y) oblique direction and (-x, -y) oblique direction. Here, by way of example, the image point identified as belonging to the cooking object is located in the x direction, (-x) direction, y direction, (-y) direction, (-x, y) oblique direction and (x, -y) oblique direction, as indicated by the solid arrows. In contrast, no image point is identified in the (x, y) oblique direction and (-x, -y) oblique direction, as indicated by the dashed arrows. This is the case for the internal point shown if the internal point is associated with the cooking object G when at least one identified image point is present in each of at least six of these directions.

[0109] The Starfill algorithm can be applied to all image points in the matrix M in one pass. In particular, the Starfill algorithm can be applied to a specific image point (e.g., first to the image point x=0, y=0) and then to the next image point, including the results for all previous image points. Already existing recognized image points are in particular associated unchanged even if they subsequently meet the conditions of the Starfill algorithm. The Starfill algorithm can be executed in particular row by row or column by column.

[0110] The Starfill algorithm is particularly suitable for filling edge contours with openings. Simple filling algorithms such as "Floodfill" - in which surfaces of consecutive pixels of one color are detected - are not suitable for non-closed contours. The Starfill algorithm is particularly suitable for images with relatively low resolution or image point matrices.

[0111] Figure 8 Shown in Figure 6 The (x, y) matrix M with image pattern BM1 is obtained after applying the Starfill algorithm once to the image pattern BM0 in , where starting from the image point x=0, y=0, the Starfill algorithm is first applied column by column in the x direction and then row by row in the y direction.

[0112] In step S8 , it is checked whether (a) the maximum number of applications or iterations of the Starfill algorithm (“number of recursions”) has been reached, or (b) there are no longer any changes in the subsequent iterations of the Starfill algorithm, depending on which condition occurs first.

[0113] If this is not the case ("N"), the process returns to step S7, otherwise ("J"), the process goes to step S9. Fig. 9 The (x, y) matrix M with image pattern BMn obtained when applying the Starfill algorithm n times (n>1) is shown.

[0114] In step S9, the method ends, wherein the image points subsequently recognized as belonging to the cooking product can serve as the basis for at least one subsequent method step S10 of a method for operating the microwave cooking device 1, for example a method for identifying a target cooking state of the cooking product G. For this purpose, the recognized image points can be used, for example, as a “mask” which is further used in one or more subsequent methods, while all image points which do not belong to the mask are not taken into account or “unmasked”.

[0115] The method described makes it possible to achieve very precise masking of the cooked food G. In particular, the method avoids only slightly heated surfaces, such as the edge of a soup plate or other cutlery parts in the exemplary embodiment.

[0116] Of course, the invention is not limited to the embodiments shown.

[0117] Generally speaking, “a”, “an” etc. can be understood as singular or plural, especially in the sense of “at least one” or “one or more”, unless explicitly excluded, for example by the expression “exactly one” etc.

[0118] A numerically specified quantity may also include exactly the specified quantity and may include the usual tolerance range unless expressly excluded.

[0119] Reference numerals list

[0120] 1 Microwave cooking equipment

[0121] 2 Cooking room

[0122] 3Cooking cabinet door

[0123] 4 Cooking racks

[0124] 5 plates

[0125] 6 Microwave generator

[0126] 7 Microwave guide

[0127] 8 Microwave feed points

[0128] 9 Rotate the antenna

[0129] 10Top of the cooking chamber

[0130] 11 Thermal imaging camera

[0131] 12 Optical digital camera

[0132] 13. Control Device

[0133] BMO Initial Image Mode

[0134] BM1 Image Mode

[0135] BMn Image Mode

[0136] DW differential thermal image

[0137] G Cooking

[0138] M Matrix

[0139] S1-S10 Method Steps

[0140] T Temperature

[0141] W1 first thermal image

[0142] W2 second thermal image

[0143] xx Direction

[0144] yy direction

Claims

1. A method (S1-S9) for identifying a cooking object (G) in at least one pixel-based thermal image (W1, W2) recorded from a cooking chamber (2) of a microwave cooking device (1), wherein - radiating microwave energy into the cooking chamber (2) (S3), - recording at least one thermal image (W1, W2) (S2, S4), - identifying image points (S6) which have a temperature value which is increased by a predetermined temperature level relative to the set of all image points, and - executing a filling algorithm which assigns image points belonging to an image area bounded by at least some of the identified image points to the culinary product (G).

2. The method (S1-S9) according to claim 1, wherein - recording a first thermal image (W1) before radiating microwave energy into the cooking chamber (2), - recording a second thermal image (W2) after radiating microwave energy into the cooking chamber (2), - creating a differential thermal image (DW), wherein the temperature difference between the temperature value recorded with the second thermal image (W2) and the temperature value recorded with the first thermal image (W1) is calculated for each image point, and - Identifying the image point in the differential thermal image (DW).

3. Method (S1-S9) according to claim 2, wherein the temperature difference is mapped onto a value range [0; 1]. 4 . The method ( S1 - S9 ) according to claim 1 , wherein image points reaching or exceeding a predefined temperature threshold value are identified as belonging to the edge of the culinary item (G).

5. The method (S1-S9) according to claims 3 and 4, wherein the temperature threshold is in the range [0.2; 0.5].

6. The method (S1-S9) according to any of the preceding claims, wherein a Starfill algorithm is used as a filling algorithm, wherein an image point is assigned to the culinary product (G) when the number of directions in which the identified image points are located, starting from an image point star, reaches or exceeds a minimum number.

7. The method (S1-S9) according to claim 5, wherein at least one of the following boundary conditions of the Starfill algorithm is set: -For corner points, the minimum number is two; -For edge points, the minimum number is three; -For interior points, the minimum number is six.

8. The method (S1-S9) according to any one of the preceding claims, wherein the filling algorithm is executed multiple times until the detected image points no longer change.

9. The method (S1-S9) according to any of the preceding claims, wherein the image points attributed to the cooked product (G) are additionally converted into image points of an optical digital camera (12) of the microwave cooking device (1).

10. The method (S1-S9) according to any of the preceding claims, wherein the identification of the position of the cooking item (G) is performed in an initial phase of a cooking process.

11. The method (S1-S9) according to claim 10, wherein during radiating microwave energy into the cooking chamber (2) to perform the method, at least one microwave operating parameter that changes the pattern image of microwaves in the cooking chamber (2) is changed.

12. A method (S1-S10) for operating a microwave cooking device (1) according to any of the preceding claims, wherein the position of the cooking item (G) is performed in a thermal image (W1, W2) according to the method (S1-S9) according to any of the preceding claims, and the cooking process is controlled based on an evaluation of the image points that are assigned to the cooking item (G).

13. A microwave cooking device (1), comprising a cooking chamber (2) that can be loaded with microwaves, a thermal image camera (11) configured to record thermal images (W1, W2) from the cooking chamber (2), and a control device (13), wherein the microwave cooking device (1) is configured to perform the method (S1-S9; S1-S10) according to any one of the above claims.

Images