Determining volumetric parameter of food to be cooked

Abstract

The invention relates to a method (S1-S14) for determining a volumetric variable of a food (6) to be cooked, which is processed in a processing chamber (1) of a household cooking appliance (2), in which method (S1, S5) images (B) in the processing chamber are recorded in chronological order, (S3) image points (BP1, BP2, BP3) belonging to the food to be cooked are identified in one image of the sequence of images, and (S4) the volume variable of the food (6) to be cooked is determined in the processing chamber (1) of the household cooking appliance (2) according to the image points (BP1, BP2, BP3). (S6) calculating the direction of movement and the speed of movement of the image points for a subsequent image by means of an optical flow method, in particular a dense optical flow method, (S8-S11) classifying the directions of movement of the previously identified image points into categories for different directions of movement, (S8-S11) calculating the number of image points falling into the respective category, and (S8-S12) determining the number of image points falling into the respective category. According to the invention, at least one quantity dependent on the volume of the food to be cooked is calculated (S12) from the number, and then, if the at least one calculated quantity satisfies at least one predetermined criterion (S14), at least one operating parameter of the household cooking device that influences the processing of the food to be cooked is varied by the household cooking device.

Description

Technical Field

[0001] The invention relates to a method for determining a volumetric parameter, in particular a volume change, of food to be cooked processed in a processing chamber of a domestic cooking appliance, wherein images of the processing chamber are recorded in chronological order, image points belonging to the food to be cooked are identified in one of the images of the image sequence, at least one parameter related to the volume of the food to be cooked is calculated, and then at least one operating parameter of the domestic cooking appliance that affects the processing of the food to be cooked is changed by the domestic cooking appliance if at least one calculated parameter meets at least one predetermined criterion. The invention also relates to a domestic cooking appliance having a processing chamber, which is arranged for thermally treating food to be cooked placed therein, at least one camera for recording images of the processing chamber, and a data processing device, which is arranged for identifying image points belonging to the food to be cooked in one of the images of the image sequence, calculating at least one parameter related to the volume of the food to be cooked from the image points, and then at least one operating parameter of the domestic cooking appliance that affects the processing of the food to be cooked is changed if at least one calculated parameter meets at least one predetermined criterion. The invention can be used in particular for baking dough, in particular bread dough. Background Art

[0002] DE 102019213638 A1 discloses a cooking device for cooking food to be cooked, in particular a household cooking device, having a heatable cooking chamber and a control device for executing at least one baking program to prepare the food to be cooked from an initial dough, and having a detection device arranged in the cooking chamber for detecting an optical sensor signal of the food to be cooked, wherein the control device is configured to execute the baking program according to the detected optical sensor signal of the food to be cooked. The control device can be used to determine the volume of the food to be cooked according to the detected optical sensor signal and to execute the baking program according to the determined volume. The control device can include an evaluation unit, which, for example, performs an image analysis to determine the volume of the food to be cooked. In particular, the control device compares the volume values ​​determined at different time points during the baking program and infers the volume change therefrom. In this way, the end time point of the dough resting or the dough proofing can be determined by analysis. The end time point can be determined according to a threshold value for the volume. Here, the threshold value can be a percentage value relative to the initial volume or an absolute value for the volume, in particular a calculated target value. Here, the threshold value can also be a volume change value per time unit, in particular a maximum volume. The control device can be configured to learn the shape of the food to be cooked based on the detected optical sensor signal, and execute the baking program based on the learned shape. Geometric parameters such as diameter, height and / or side length are preferably also learned here.

[0003] DE 102019107859 A1 discloses a method for operating a cooking device, which has a cooking area and a processing device for preparing food to be cooked. During the cooking process, the food to be cooked is monitored in the cooking area. In this case, during the cooking process, a measure of the size of the food to be cooked is repeatedly determined by means of a camera device. This determines a measure of the change in size of the food to be cooked over time. Based on the measure of the change in size, the food to be cooked is assigned to at least one group of food to be cooked. From the area extension of the image element assigned to the food to be cooked, the basic area of ​​the food to be cooked can be at least approximately determined, and from the distance difference between the image element assigned to the food to be cooked and the image element adjacent to the food to be cooked but not assigned to the food to be cooked, the height of the food to be cooked can be at least approximately determined, and from the basic area and the height, the volume of the food to be cooked can be at least approximately determined, and the change in volume over time can be used as a measure of the change in size of the food to be cooked. In this case, the time course of the estimated value of the volume of the food to be cooked can be continuously recorded at the beginning of cooking and during cooking. The electronic device can continuously analyze the temporal trend of the estimated value of the volume of the food to be cooked in cooperation with the selected cooking program and intervene in the regulation of the cooking program. This can be done, for example, by changing the temperature of the cooking chamber, assisting in a dialogue with the user when the cooking time is about to end, or autonomously ending the cooking process. The electronic device then intervenes in the cooking process based on this analysis or information, for example by switching off the heating and other energy supplies, by keeping warm or (fast) cooling if necessary.

[0004] DE 102019107812 A1 discloses a method for operating a cooking device with a cooking zone for preparing food to be cooked, wherein the food to be cooked is monitored in the cooking zone during the cooking process and images of the cooking zone are captured over time by means of a camera device, wherein each image consists of a plurality of image elements and is evaluated by means of a processing device. In this case, image elements that change over time are identified and assigned as belonging to the food to be cooked so as to be distinguishable from image elements from outside the food to be cooked. In this case, use is made of the fact that image elements containing the food to be cooked and image elements containing the cooking zone with cooking chamber walls, containers, etc. behave completely differently. This difference is not only related to changes caused by browning and discoloration, but also to distances caused by changes in volume or surface temperature. The images captured by means of the camera device can contain spatial image information from the cooking zone, wherein, in order to identify image elements that change over time, at least one temporal change of the spatial image information, preferably a change in distance from the camera device, is analyzed.

[0005] DE 102019212364 A1 discloses a method for operating a domestic cooking appliance having a cooking chamber and at least one camera for recording pixel-based images from the cooking chamber, wherein (a) at least one image from the cooking chamber is recorded with the aid of the at least one camera, and (b) the image is evaluated without the brightness values ​​of its associated pixels. Summary of the invention

[0006] The object of the present invention is to at least partially overcome the disadvantages of the prior art and in particular to provide an easy-to-implement, robust method for calculating a variable related to the volume of a heat-treated food to be cooked from a sequence of images.

[0007] This object is achieved according to the features of the independent claims. Preferred embodiments can be found in particular in the dependent claims.

[0008] This object is achieved by a method for determining a volume variable of food to be cooked in a cooking chamber of a domestic cooking appliance, wherein:

[0009] - Time-sequential images of the process chamber,

[0010] - identifying in an image of the image sequence, in particular the first image, an image point belonging to the food to be cooked,

[0011] - calculating the direction and speed of movement of the image points for subsequent images by means of an optical flow method, in particular a dense optical flow method,

[0012] - classifying the previously identified directions of motion of the pixels into categories for different directions of motion,

[0013] - count the number of pixels that fall into the corresponding class,

[0014] - calculating from these numbers at least one parameter related to the volume of the food to be cooked,

[0015] - Then, at least one operating parameter of the domestic cooking appliance influencing the processing of the food to be cooked is changed by the domestic cooking appliance if the at least one calculated variable meets at least one predetermined criterion.

[0016] The advantage of the method is that the volume of the thermally treated food to be cooked can be reliably determined independently of the size of the food to be cooked, the positioning of the food to be cooked and the type of container for the food to be cooked.

[0017] The domestic cooking appliance can be, for example, an oven, a steam treatment device, a microwave oven, a kitchen multifunction appliance or any combination thereof, for example an oven with a steam treatment function, in particular a steam cooking function. The treatment chamber can also be called a cooking chamber, even if the food to be cooked is not cooked therein, but is, for example, treated therein, wherein the food to be cooked is moistened at room temperature.

[0018] The method is particularly advantageous if the food to be cooked undergoes a significant volume change at least during a time period of its processing. In a further development, the food to be cooked is a dough, for example, consisting of bread dough or the like.

[0019] For example, the time sequence may include taking images at intervals of a few seconds to a few minutes, such as every 5 seconds, 10 seconds, 30 seconds, 1 minute, 2 minutes, etc.

[0020] The recording of the image in the processing chamber includes in particular also showing the food to be cooked which is located in the processing chamber in the image.

[0021] In one refinement, at least part of the image is recorded by means of a camera of the domestic cooking appliance. The camera can be arranged, for example, in the region of a cover, a side wall or a rear wall of a processing chamber (also referred to as a muffle furnace) or on a door closing a charging opening of the processing chamber. In one refinement, the camera is directed obliquely toward the processing chamber, which facilitates the capture of vertical movements of the food to be cooked. The camera can also be referred to as a cooking chamber camera. The camera is in particular a pixel-based digital camera, in particular a color digital camera.

[0022] The domestic cooking appliance may have one or more cooking chamber cameras. The method may be performed based on the images of only one camera or based on the images of multiple cameras.

[0023] Identifying a sequence of pixels belonging to food to be cooked can also be referred to as segmentation. For example, identification can be performed by object marking, brightness determination and / or color differences (e.g., dark cooking chamber walls, food racks and food containers to be cooked, light-colored food to be cooked).

[0024] The image in which the identification pixel is located can in particular be the first image in a sequence of images for carrying out the method.

[0025] Optical flow methods, in particular dense optical flow methods, are well known. In this case, the movement of an object present in an image is detected by tracking image points representing the object in an image sequence, in particular from one image to another. For each of the image points on the image plane, a direction of movement, in particular in polar coordinates (using polar angles), can be obtained for each of the image points on the image plane. The image point is represented by the radius r and the speed of movement (represented by the radius r). This is equivalent to assigning a vector on the image plane to the corresponding image point, where the direction of the vector corresponds to the direction of movement of the image point and the length of the vector corresponds to the speed of movement. This movement can last for more than two images. In particular, the movement direction of the image point on the image plane can be tracked for multiple images.

[0026] The movement of previously identified (“segmented”) pixels is divided into classes for different directions of movement, which in particular includes that only the movement of previously identified pixels is divided into classes for different directions of movement.

[0027] In one development, the optical flow method is applied only to previously identified pixels belonging to the food to be cooked. In a particularly advantageous development, the optical flow method is applied to all pixels and only the pixels identified as belonging to the food to be cooked are then further considered. In one development, the pixels identified for all images of the image sequence used correspond to the pixels identified in the first image.

[0028] The classification of the pixel movements in different directions of movement includes, in particular, a check whether the direction of movement determined by the optical flow method has a certain relationship with at least one predetermined direction of movement. If this is the case, the relevant pixel is assigned to a class belonging to this direction of movement, otherwise not. It is thus possible that previously identified pixels are not assigned to at least one of the classes, but instead - for the entire processing sequence or time segment - remain in the "no class" state.

[0029] An improved solution is to perform non-weighted classification on the pixels, that is, each pixel has the same unit value, such as "1". Therefore, the number of a category is exactly the same as the number of pixels in the category.

[0030] One improvement is to classify the image points in a non-weighted manner, i.e., to assign a weighting coefficient to each image point, thereby assigning a "weight number" to each image point. Therefore, different image points contribute to different degrees to the (total number) in the category. One improvement is that the weighting depends on the movement speed of the corresponding image point. For example, the faster the movement speed, the greater the weighting coefficient and the greater the weighting number. In particular, the weighting coefficient can be a function of the movement speed, and in particular, it can be proportional to the movement speed.

[0031] In one refinement, the different movement directions include movement in opposite directions along the same axis. In one refinement, the different movement directions include movement in opposite directions along multiple axes.

[0032] Calculating at least one parameter related to the volume of the food to be cooked from the number particularly includes deriving or estimating the parameter from the absolute size and / or relationship of the numbers of at least two categories. This association can also include the number of unclassified pixels. For example, this association can include the relationship between the number of different categories and / or unclassified pixels, the relationship with a threshold value, etc. For example, the association between the number and the at least one parameter can be determined by experiments, simulations and / or machine learning. For example, this association can be stored in the form of a table or function in the household cooking device, in which the number is used as a variable.

[0033] The satisfaction of at least one predetermined criterion may include the satisfaction of a condition or relationship, the satisfaction of one condition in a set of multiple conditions, and / or the simultaneous satisfaction of at least two conditions in a set of multiple conditions. For example, the criterion may include reaching, exceeding, and / or falling below at least one predetermined threshold.

[0034] In one refinement, the criterion includes at least one association of a number assigned to the respective class and, if applicable, at least one association of a number of pixels without a class. Thus, in this refinement, no volume-dependent physical variable is explicitly calculated, but at least one volume-dependent variable is implicitly or similarly given by the association of the number.

[0035] In one embodiment, the different directions of movement include a movement in the direction of gravity and a movement against the direction of gravity, i.e. a movement in the opposite direction of movement along the axis of gravity ("gravity axis"). This has the advantage that it is possible to check in a particularly easy way whether the food to be cooked is moving upwards or rising (hereinafter also referred to as "rising") or moving downwards or falling (hereinafter also referred to as "falling"). This also makes it particularly easy to quantitatively determine the rise and fall. The gravity axis is very close to a vertical, in particular downward-pointing, spatial axis. The gravity axis can be determined in the camera image by its geometrical arrangement and orientation, regardless of whether there is food to be cooked.

[0036] In one embodiment, if an image point moves against the direction of gravity within a first predetermined angle deviation from the gravity axis, it is classified into the first category (corresponding to rising); if an image point moves toward the direction of gravity within a second predetermined angle deviation from the gravity axis, it is classified into the second category (corresponding to falling). This has the advantage of providing a particularly simple classification and the result is reliable.

[0037] The image point moves within a predetermined angle deviation from a determined reference axis, in particular, includes that the angle deviation of the movement direction of the image point on the image plane from the reference axis is not greater than or less than the predetermined angle deviation. The angle deviation may be symmetrical or asymmetrical about the axis, that is, the magnitude of the clockwise angle deviation and the magnitude of the counterclockwise angle deviation may be the same or different.

[0038] An improvement is that if the clockwise angle deviation from the downward gravity axis is or] [angle range, then the image point is classified into the first category, where the applicable boundary conditions are and

[0039] choose and / or select This is a particularly advantageous improvement for the reliable and robust determination of volume-related variables. (symmetrical angular deviation) or (Asymmetric angle deviation) can be applied.

[0040] An improvement is that if the clockwise angle deviation from the downward gravity axis is or] [, the image point is classified into the second category, where the applicable boundary conditions are and

[0041] choose and / or select This is a particularly advantageous improvement for the reliable and robust determination of volume-related variables. (symmetrical angular deviation) or (Asymmetric angle deviation) can be applied.

[0042] For example, if the reference axis is the downward gravity axis, and the first and second angular deviations are symmetrical, for example, numerically 20°, then the image points whose movement direction is in the angular range of [-20°; +20°] or ]-20°; +20°[ with the gravity axis are classified into the first category, and the image points whose movement direction is in the angular range of [160°; 200°] or ]160°; 200°[ with the gravity axis are classified into the second category.

[0043] In one embodiment, the different movement directions include movement away from the center of gravity of the surface of the food to be cooked and movement toward the center of gravity of the surface of the food to be cooked. This has the advantage that, in addition to or instead of rising and falling, lateral or transverse expansion (hereinafter referred to as "expansion") of the food to be cooked and transverse contraction (hereinafter referred to as "contraction") of the food to be cooked can also be considered in order to calculate at least one parameter related to the volume of the food to be cooked.

[0044] In one development, the surface centroid corresponds to the geometrical centroid of the segmented image points in the image plane.

[0045] In one embodiment, the center of gravity of the surface of the food to be cooked is determined, and a corresponding connecting line with the center of gravity of the surface is determined for the image point belonging to the food to be cooked, and if the image point moves away from the center of gravity of the surface within a third predetermined angle deviation from the connecting line, the image point is classified into the third category (corresponding to expansion), and if the image point moves toward the center of gravity of the surface within a fourth predetermined angle deviation from the connecting line, the image point is classified into the fourth category (corresponding to contraction). This has the advantage that reliable measurements of expansion and contraction of the food to be cooked can be provided in a particularly simple manner.

[0046] The image point moves away from the surface center of gravity within the predetermined angle deviation from the connecting line, in particular, the angle deviation between the movement direction of the image point on the image plane and the connecting axis from the surface center of gravity to the image point is not greater than or less than the third predetermined angle deviation. The image point moves toward the surface center of gravity within the predetermined angle deviation from the connecting line, in particular, the angle deviation between the movement direction of the image point on the image plane and the connecting axis from the surface center of gravity to the image point is not greater than or less than the fourth predetermined angle deviation. The angle deviation can be symmetrical or asymmetrical about the connecting axis.

[0047] An improved solution is that if the angular deviation in the clockwise direction from the connecting axis pointing away from the center of gravity of the surface is or , then the image point is classified into the third category, where the applicable boundary conditions are and

[0048] choose and / or select This is a particularly advantageous improvement for the reliable and robust determination of volume-related variables. (symmetrical angular deviation) or (asymmetric angle deviation) can be applied. Generally speaking, (symmetrical angular deviation) or (asymmetric angle deviation) can be applied. A particularly advantageous improvement is that Applicable, i.e. the image point only works if the angular deviation in its direction of motion is within the angular range It is classified into the third category only when it is within the angle range, for example, within the angle range [0°; 30°].

[0049] An improved solution is that if the angular deviation from the connecting axis in the clockwise direction is or] [angle range, then the image point is classified into the fourth category, where the applicable boundary conditions are and

[0050] choose and / or select This is a particularly advantageous improvement for the reliable and robust determination of volume-related variables. (symmetrical angular deviation) or (asymmetric angle deviation) can be applied. A particularly advantageous improvement is that Applicable, i.e. the image point only works if the angular deviation in its direction of motion is within the angular range It is classified into the fourth category only when the angle is within the range of [150°; 180°].

[0051] For example, if the third and fourth angular deviations about the connecting axis serving as the reference axis are symmetrical, for example numerically predetermined to be 25°, then image points whose direction of movement is within the angular range of [-25°; +25°] or ]25°; +25°[ in the direction of the connecting axis pointing away from the center of gravity of the surface will be classified into the third category, while image points whose direction of movement is within the range of [160°; 200°] or ]160°; 200°[ relative to the connecting axis will be classified into the fourth category.

[0052] One embodiment is to assign a pixel to a certain class only if its speed is above a predetermined threshold value. This makes the determination of the volume-related variable more robust, since the classified pixel must have a minimum speed, so that pixels that are barely moving are not taken into account.

[0053] In one development, the movement speed corresponds to the scalar length of the movement vector of the corresponding pixel.

[0054] In one development, the image points in the image that belong to the food to be cooked are identified based on their color. This is particularly easy to implement and is based on the knowledge that the color of the food to be cooked is usually very different from the dark blue, dark gray or black cooking chamber walls, the mostly black baking trays, the mostly silver metal grids and the cooking containers. This applies in particular to fresh dough, which is usually light brown in color. The image points whose color coordinates are in a subspace of the color space of the image that is predetermined for the color of the placed food to be cooked are assigned to the food to be cooked, and in particular other image points are not assigned to the food to be cooked.

[0055] One improvement is that the captured image is an RGB image, that is, the possible colors of the image points are in the RGB color space.

[0056] A particularly easy-to-implement improvement is to identify image points belonging to the food to be cooked in the image by the fact that the H value associated with the HSV (“hue, saturation, value”) color coordinate system is within a predetermined value range. The advantage of this is that only one coordinate axis of the color space, namely the position of the image point on the H axis, is required to reliably determine the belonging of the image point to the light brown food to be cooked. This utilizes the fact that the brown tone is located on the H axis in the HSV color space, which is particularly advantageous for brown food to be cooked, especially light brown food to be cooked, especially dough.

[0057] Thus, for example, if the value of an image point belonging to dough on the H axis ("H value") is within a value range defined by a lower threshold ("H lower threshold") and an upper threshold ("H upper threshold"), which value range includes the brown tone of a typical dough, the image point can be identified.

[0058] If the color space of the captured image is not the HSV color space (eg, an RGB image), the color coordinates of the image points may be converted or transformed from the original color space into the HSV color space.

[0059] As an alternative to the HSV color space, the HSL (“hue, saturation, lightness”) color space—which is particularly suitable with regard to limit values—can also advantageously be used.

[0060] In one embodiment, the at least one parameter related to the volume of the food to be cooked comprises at least one parameter from the group consisting of: the volume of the food to be cooked, the change in the volume of the food to be cooked over time and / or the speed of change of the volume of the food to be cooked.

[0061] In one embodiment, at least one predetermined criterion comprises reaching a maximum volume of the food to be cooked and / or reaching a minimum change in the volume of the food to be cooked. This has the advantage that reaching an extreme value and / or approaching a static volume value, in particular reaching a maximum volume of the food to be cooked and / or reaching a minimum change in volume, can be particularly well identified. For example, if a change in volume, in particular a volume increase, of the food to be cooked between two consecutive images is less than a predetermined threshold, this can be considered a minimum change in volume.

[0062] In one embodiment, the at least one operating parameter influencing the processing of the food to be cooked comprises at least one operating parameter from the group consisting of: temperature in the processing chamber (also referred to as "cooking chamber temperature") and / or humidity in the processing chamber. This has the advantage that the processing of the food to be cooked can be adapted particularly reliably and promptly to the occurrence of certain states of the food to be cooked, thereby improving the cooking result, in particular in the case where the food to be cooked is sensitive to the cooking chamber temperature and / or humidity in the processing chamber due to changes in volume.

[0063] In one embodiment, the food to be cooked is a dough. The method is particularly applicable to such food to be cooked, since in the case of dough, its volume and / or its volume change is a particularly reliable indicator of the presence and / or completion of certain processing stages. If the processing stages, in particular their end, can be determined more reliably, the preparation results of the food to be cooked will also be significantly improved.

[0064] In one refinement, the dough is bread dough, since in the case of bread dough the volume or a dimension derivable from the volume can be detected particularly well and bread dough has various processing stages which exhibit significantly different volume behavior and require different ambient conditions to achieve particularly good results.

[0065] When baking bread, the dough expands in a first phase at high humidity and high cooking cabinet temperatures, increasing its volume considerably (the so-called "dough expansion"). This then transitions to a second phase (the so-called "finishing" or "browning"), in which the bread dough essentially maintains its volume or even slightly decreases, while producing a browned crust. The second phase usually uses lower temperatures and lower humidity in the cooking cabinet than the first phase and also lasts longer than the first phase. The more precisely the transition point between the two phases can be determined in order to adjust the cooking cabinet atmosphere (in particular the cooking cabinet temperature, but also the humidity and / or the heating elements used at the time), the better the baking result.

[0066] One embodiment is to reduce the temperature and / or humidity of the treatment chamber when the maximum volume of the dough and / or the minimum change in the dough volume are reached. In this way, particularly good results can be achieved when transitioning from the dough proofing stage to the browning stage.

[0067] The object is also achieved by a domestic cooking appliance, which has:

[0068] - a processing chamber, which is designed to thermally treat the food to be cooked placed therein,

[0069] - at least one camera for capturing images of the process chamber, and

[0070] - a data processing device, arranged to

[0071] - identifying in an image of the image sequence an image point belonging to the food to be cooked,

[0072] - calculating the direction and speed of movement of the image points for subsequent images by means of an optical flow method, in particular a dense optical flow method,

[0073] - classify the previously identified motion of the pixels into classes for different directions of motion,

[0074] - count the number of pixels that fall into the corresponding class,

[0075] - calculating from these numbers at least one parameter related to the volume of the food to be cooked,

[0076] - Then, at least one operating parameter of the domestic cooking appliance influencing the processing of the food to be cooked is changed if at least one calculated variable meets at least one predetermined criterion.

[0077] The domestic cooking device can be constructed similarly to the method, and vice versa, and has the same advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] The above characteristics, features and advantages of the present invention and the modes and methods for achieving the same will become clearer and easier to understand through the following schematic description of the embodiments described in detail with reference to the accompanying drawings.

[0079] Figure 1 An image taken from a processing chamber is shown showing the surface of bread dough plotted into a number of coordinate systems;

[0080] Figure 2 Shows Figure 1 The image in , with the barycentric axis;

[0081] Figure 3 Shows Figure 1 with axes connecting the surface centroids and image points; and

[0082] Figure 4 A possible method sequence for carrying out the method is shown. DETAILED DESCRIPTION

[0083] Figure 1 A color image B is shown which was recorded by means of a digital camera 7 from a processing chamber in the form of a cooking chamber 1 of a domestic cooking appliance 2. The domestic cooking appliance 2 can be, for example, an oven with an additional steam processing function, such as so-called "additional steam". The image B shows a cooking chamber wall 3, a wire rack 4, a box-shaped baking pan 5 placed on the wire rack 4, and dough 6, in particular bread dough, placed in the baking pan 5. The image B generally has p pixels in its x-axis direction or along its x-axis x and q pixels in its y-axis direction or along its y-axis y, for example 640 x 480 pixels, 1024 x 768 pixels, etc. The x-axis x and the y-axis y intersect at the geometrical surface centroid SP of the pixels belonging to the dough 6.

[0084] Also drawn in are the Cartesian coordinate axes x' and y', which also intersect at the centroid SP of the geometrical surface, the x' axis corresponding to the centroid axis pointing downwards, which can therefore also be referred to as "g". The centroid axis g is the same for all image points. The centroid axis x' or g results from the arrangement and orientation of the camera 7 and is therefore independent of whether there is dough 6 or not. The (x', y') coordinate system is rotated relative to the (x, y) coordinate system.

[0085] Also drawn in are the Cartesian coordinate axes x" and y", which also intersect at the geometrical surface center SP, wherein the x" axis is oriented in the longitudinal direction of the dough 6. The advantage of choosing this coordinate system is that the movement of the image point relative to the food to be cooked can be set particularly easily.

[0086] The domestic cooking device 2 also has a data processing device 8 which is coupled to the camera 7 in terms of data technology, can control the camera 7 for image recording, and can also be set, for example programmed, to process the images B transmitted by the camera 7, for example to calculate a variable related to the volume of the dough 6, such as a volume change over time. The data processing device 8 can also be set to change at least one operating parameter that influences the processing of the dough 6, such as cooking chamber temperature, humidity, etc. In particular, the data processing device 8 can be a central control device of the domestic cooking device 2.

[0087] Figure 2 Shows Figure 1 The image B in FIG. 6 is a graph of the image B in which the centroid axis g passes through the image point BP1 assigned to the dough 6. The motion vector v(BP1) calculated from the temporally consecutive images B by the dense optical path method can be assigned to the image point BP1. The motion vector v(BP1) can be expressed as the scalar product of its nominal motion direction and its length, in particular the polar coordinates r (length) and (angle with respect to the axis of gravity g).

[0088] In addition, a first angle range [160°; 200°] in the direction of gravity or ±20° against gravity (corresponding to the direction against gravity g) is drawn in. If the motion vector v(BP1) is within the first angle range, it can be considered that the dough 6 is raised at the position of the image point BP1.

[0089] In addition, a second angle range [340°; 20°] or ±20° (corresponding to the direction along the gravity axis g) is also shown. If the motion vector v(BP1) is within the second angle range, it can be considered that the dough 6 falls at the position of the image point BP1.

[0090] Figure 3 Shows Figure 1The image B in FIG. 6 has a connection axis c(BP1) between the surface center of gravity SP and the image point BP2, and a connection axis c(BP2) between the surface center of gravity SP and the image point BP3 of the dough 6. The connection axes c(BP2) and c(BP3) point from the surface center of gravity SP to the corresponding image point BP2 or BP3, respectively.

[0091] The motion vectors v(BP2) and v(BP3) of the image point BP2 or BP3 are also drawn. The motion vector v(BP2) points away from the surface centroid SP within the drawn third angle range, for example, along the connecting axis c(BP2)±20°, while the motion vector v(BP3) points to the surface centroid SP within the drawn fourth angle range, for example, in the direction opposite to the connecting axis c(BP3)±20°. This can be interpreted as the dough expanding at the position of the image point BP2 but contracting at the position of the image point BP3.

[0092] It is possible that none, some or all of the pixels assigned to the dough 6 have a motion vector v within the first or second angular range and / or within the third or fourth angular range. It is also possible that none, some or all of the pixels assigned to the dough 6 have a motion vector v that is not within any angular range.

[0093] Figure 4 A possible method sequence for carrying out the bread baking method is shown.

[0094] After the dough 6 has been placed in the cooking chamber 1, the bread baking process begins, for example, under the control of the data processing device 8. In a first step S1, for example, a first RGB image B of the cooking chamber is captured under the control of the data processing device 8, for example, with the help of a color digital camera 7. 1 , including a wire rack 4, a baking tray 5 and dough 6, such as similar to Figure 1 Image B in .

[0095] In step S2, image B 1 The RGB color coordinates of are converted into HSV (hue, saturation, value) color coordinates by the data processing device 8, for example.

[0096] In a third step S3, for example with the aid of the data processing device 8, those pixels are identified whose H value is within a value range defined by a lower threshold value ("H lower threshold value") and an upper threshold value ("H upper threshold value"), which value range corresponds in particular to the light brown color of the fresh dough 6. The result of this "segmentation" is an image mask in which all detected pixels are assigned a certain information, for example "1", while all other pixels are assigned another information, for example "0". It is assumed here that only the pixels belonging to the dough 6 are within the predetermined value range. These pixels can also be referred to as "segmented" pixels.

[0097] An improved solution is to start from the first image B 1 The image mask is determined once in , and then remains unchanged for the rest of the process.

[0098] In a fourth step S4 , the surface center of gravity SP, for example the geometric center of gravity, which is assigned to the segmented image points of the dough 6 is determined or calculated, for example by means of the data processing device 8 .

[0099] In a fifth step S5, a further image B is recorded, for example controlled by the data processing device 8. i , where i=2.

[0100] In a sixth step S6, a dense optical flow method is applied to the image B, for example by means of the data processing device 8. i and B i-1 First, the motion vector v obtained for each of the image points on the (x, y) image plane is converted from the Cartesian coordinate system to a motion vector v in the polar coordinate system, whose length r represents its motion speed and its polar angle The result is a motion matrix with x·y motion vectors v, which are assigned to pixels at the same matrix position. Therefore, the motion of each pixel can be estimated using the motion matrix.

[0101] In a seventh step S7, the image mask calculated in step S3 is applied to the motion matrix obtained in step S6, for example by means of the data processing device 8, in particular by scalar multiplication of the corresponding pixels, so that only the motion vectors assigned to the pixels of the dough 6 are retained, while the remaining pixels are not assigned any motion vectors or are assigned motion vectors of length zero. In other words, a motion matrix is ​​generated in which only the motion vectors of the pixels assigned to the dough 6 are taken into account.

[0102] In step S8, the segmented image points are checked, for example by means of the data processing device 8, to see whether their direction of movement is within a first predetermined angle range relative to the gravity axis g, i.e. whether these image points have risen. If this is the case, the image points are classified into the first class or assigned to the first class. The number n of all such image points is obtained from step S8. 1 .

[0103] In step S9, the segmented image points are checked, for example by means of the data processing device 8, to see whether their direction of movement is within a second predetermined angle range in the direction of the gravity axis g, i.e., whether these image points have fallen. If this is the case, the image points are classified into the second category or assigned to the second category. The number n of all such image points is obtained from step S9 2 .

[0104] In step S10, the segmented image points are checked, for example by means of the data processing device 8, to see whether their movement direction is within a third predetermined angular range along the connecting axis between the surface center of gravity SP and the corresponding image point, i.e. whether these image points have expanded. If this is the case, the image points are classified into the third category or assigned to the third category. The number n of all such image points is obtained from step S10 3 .

[0105] In step S11, the segmented image points are checked, for example by means of the data processing device 8, to see whether their direction of movement is within a fourth predetermined angular range opposite to the connecting axis between the surface center of gravity SP and the corresponding image point, i.e., whether these image points have contracted. If this is the case, the image points are classified into the fourth class or assigned to the fourth class. The number n of all such image points is obtained from step S10 4 .

[0106] In step S12, for example, by means of the data processing device 8, from the number n 1 and n 2 and / or from the number n 3 and n 4 Determine image B i-1 and B i The volume change ΔV between image captures.

[0107] In step S13 , it is checked, for example by means of the data processing device 8 , whether the volume change ΔV satisfies at least one specific criterion, for example according to ΔV<ΔV thr Less than or according to ΔV≤ΔV thr Less than or equal to the predetermined positive threshold ΔV thr , where ΔV=0 or ΔV<0 applies. These criteria correspond to a state of the dough 6 in which the volume V of the dough has hardly increased or has even slightly shrunk, which indicates that the dough has transitioned from fermentation to browning.

[0108] If the volume change ΔV does not satisfy at least one criterion (“No”), the process returns to step S5 and generates another image B. i , where i:=i+1. If you are shooting image B i and B i+1 An interval of about 10 seconds is particularly advantageous.

[0109] However, if the volume change ΔV meets at least one criterion ("yes"), then in step S14, at least one operating parameter of the household cooking appliance that affects the processing of the dough 6 is changed, for example with the aid of the data processing device 8, such as lowering the cooking chamber temperature, reducing the air humidity, etc., and depending on the type of food to be cooked, in particular the type of dough 6, for example whether different types of bread dough, croissant dough, etc. are used in an improved solution.

[0110] The rotation matrix can be used to calculate the angular difference between the image point motion vector v and the associated reference axis g or c.

[0111] Of course, the invention is not limited to the exemplary embodiments shown.

[0112] For example, steps S8 to S11 may be performed in any order.

[0113] The angle range can also be fixedly preset or—for example, depending on the operating program used or the known food to be cooked—can be variably adjusted.

[0114] Furthermore, the length of the motion vector |r| can also be selectively taken into account, for example by selecting a vector with a predetermined minimum length |r| min Or the length projected onto the corresponding reference axis (shear force axis g, connection axis c) | r proj | has a predetermined minimum length | r proj | min In one refinement, only pixels with sufficient length are classified. In another refinement, pixels without sufficient length are also initially classified and then deleted from the category again at the end of steps S8 to S11.

[0115] In addition, the number of pixels n that are not classified into any category 5 It can also be used to determine image B i-1 and B i The volume change ΔV between image captures.

[0116] Generally speaking, "one", "an", etc. can be understood as singular or plural, especially "at least one" or "one or more", etc., as long as there is no explicit exclusion, such as "exactly one", etc. Numerical indications can also include the exact specified number and the usual tolerance range, as long as there is no explicit exclusion.

[0117] Reference numerals list

[0118] 1 Cooking room

[0119] 2 Home cooking equipment

[0120] 3 Cooking cabinet wall

[0121] 4 Metal Grid

[0122] 5 baking sheet

[0123] 6 Dough

[0124] 7 Camera

[0125] B-image

[0126] BP1 pixel

[0127] BP2 pixel

[0128] BP3 pixel

[0129] c Connecting shaft

[0130] SP Center of Gravity

[0131] S1-S14 Method Steps

[0132] v Motion vector

[0133] xx axis

[0134] x'x' axis

[0135] x"x" axis

[0136] yy-axis

[0137] y'y' axis

[0138] y"y" axis

[0139] g gravity axis

Claims

1. A method (S1-S14) for determining a volume parameter of food to be cooked (6) processed in a processing chamber (1) of a domestic cooking device (2), wherein: - (S1, S5) taking images (B) in the processing chamber (1) in chronological order, (S3) identifying, in an image (B) of the image sequence, image points (BP1, BP2, BP3) belonging to the food to be cooked (6), (S6) calculating the movement direction and movement speed of the image points (BP1, BP2, BP3) for the subsequent image (B) by means of an optical flow method, in particular a dense optical flow method, - (S8-S11) classifying the movement directions of the previously identified image points (BP1, BP2, BP3) into categories for different movement directions, - (S8-S11) Calculate the number of image points (BP1, BP2, BP3) that fall into the corresponding category, - (S12) calculating from said number at least one parameter related to the volume of the food (6) to be cooked, - Then, (S13) if at least one calculated variable meets at least one predetermined criterion, (S14) at least one operating parameter of the domestic cooking appliance (2) influencing the processing of the food (6) to be cooked is changed by the domestic cooking appliance (2).

2. The method (S1-S14) according to claim 1, wherein: The different directions of movement include - movement in the direction of gravity (g) and -Movement against the direction of gravity (g).

3. The method (S1-S14) according to claim 2, wherein: Image point (BP1) - (S8) if the movement is against the direction of gravity within a first predetermined angular deviation from the gravity axis (g), it is classified as the first category; - (S9) If the movement is in the direction of gravity within a second predetermined angular deviation from the gravity axis (g), it is classified as the second category.

4. The method (S1-S14) according to any one of the preceding claims, wherein: The different directions of movement include - (S10) movement away from the center of gravity (SP) of the surface of the food to be cooked (6) and - (S11) Movement towards the centre of gravity (SP) of the surface of the food to be cooked (6).

5. The method (S1-S14) according to claim 4, wherein: The surface center of gravity (SP) of the food to be cooked (6) is determined and corresponding connecting lines with the surface center of gravity (SP) are determined for the image points (BP1, BP2, BP3) belonging to the food to be cooked (6), and the image points (BP2, BP3) - (S10) if it (BP2) moves away from the center of gravity of the surface within a third predetermined angular deviation from the connecting line c (BP2), it is classified into the third category; -(S11) If it (BP3) moves toward the surface center of gravity (SP) within a fourth predetermined angular deviation range from the connecting line c (BP3), it is classified into the fourth category.

6. The method (S1-S14) according to any one of the preceding claims, wherein: Only when the movement speed of a pixel (BP1, BP2, BP3) is higher than a predetermined threshold value is it assigned to a specific class.

7. The method (S1-S14) according to any one of the preceding claims, wherein: (S3) The image points (BP1, BP2, BP3) belonging to the food (6) to be cooked in the image (B) are identified by the fact that the H values ​​associated with the HSV color coordinate system are within a predetermined value range.

8. The method (S1-S14) according to any one of the preceding claims, wherein: The at least one parameter related to the volume of the food (6) to be cooked comprises at least one parameter from the following group: - the volume of the food to be cooked (6), - the change in volume of the food to be cooked (6), - the rate of change of the volume of the food to be cooked (6).

9. The method (S1-S14) according to any one of the preceding claims, wherein: At least one predetermined criterion includes - the maximum volume of the food to be cooked (6) is reached and / or - Achieving minimal changes in the volume of the food (6) to be cooked.

10. The method (S1-S14) according to any one of the preceding claims, wherein: The at least one operating parameter affecting the processing of the food to be cooked (6) comprises at least one operating parameter from the group consisting of: - the temperature in the process chamber (1), - Humidity in the process chamber (1).

11. The method (S1-S14) according to claims 8 and 9, wherein: The food (6) to be cooked is dough, in particular bread dough, and (S13) when the maximum volume of the dough and / or the minimum change in the volume of the dough is reached, (S14) the temperature and / or humidity in the processing chamber (1) is reduced (S14).

12. Household cooking equipment (2) having: a processing chamber (1) which is designed to thermally treat the food to be cooked (6) placed therein, - at least one camera (7) for capturing images of the process chamber (1), and - a data processing device (8) arranged to - identifying in an image of the image sequence (B) image points (BP1, BP2, BP3) belonging to the food to be cooked (6), - calculating the direction and speed of movement of the image points (BP1, BP2, BP3) for the subsequent image (B) by means of an optical flow method, in particular a dense optical flow method, - classifying the movement directions of the previously identified pixels (BP1, BP2, BP3) into categories for different movement directions, - Count the number of pixels (BP1, BP2, BP3) that fall into the corresponding category, - calculating from said number at least one parameter related to the volume of the food (6) to be cooked, - Then, if at least one calculated variable meets at least one predetermined criterion, at least one operating parameter of the domestic cooking appliance (2) that influences the processing of the food (6) to be cooked is changed.

Images