DETERMINING A VOLUME SIZE OF GARGUT

Description

[0001] The invention relates to a method for determining a volume quantity, in particular volume change, of food being cooked in a treatment chamber of a household cooking appliance, in which images from the treatment chamber are taken in temporal sequence, pixels belonging to the food being cooked are identified in an image of the image sequence, at least one quantity relating to the volume of the food being cooked is calculated and then, if the at least one calculated quantity fulfills at least one predetermined criterion, at least one operating parameter of the household cooking appliance influencing the treatment of the food is varied by the household cooking appliance.The invention also relates to a household cooking appliance comprising a treatment chamber configured for heat-treating food placed therein, at least one camera for capturing images from the treatment chamber, and a data processing device configured to identify pixels belonging to the food in a sequence of images, to calculate at least one quantity relating to the volume of the food from the pixels, and then, if the at least one calculated quantity fulfills at least one predetermined criterion, to vary at least one operating parameter of the household cooking appliance that influences the treatment of the food. The invention is particularly advantageously applicable to baking dough pieces, especially bread dough.

[0002] DE 10 2019 213 638 A1 discloses a cooking appliance, in particular a household cooking appliance, for cooking food, comprising a heated cooking chamber and a control device for executing at least one baking program for preparing the food from a starting dough, and a detection device arranged in the cooking chamber for detecting an optical sensor signal of the food, wherein the control device is configured to execute the baking program depending on the detected optical sensor signal of the food. The control device can be configured to determine the volume of the food depending on the detected optical sensor signal and to execute the baking program depending on the determined volume. The control device can include an evaluation unit that, for example, performs an image analysis to determine the volume of the food.In particular, the control device compares volume values ​​measured at different times during the baking program and thus infers a change in volume. In this way, the end time of a dough resting or proofing phase can be determined analytically. An end time can be determined based on a volume threshold. This threshold can be a percentage value relative to an initial volume, an absolute value, or, in particular, a calculated target volume. The threshold can also refer to the volume change per unit of time and, in particular, represent a volume maximum. The control device can be configured to determine the shape of the baked goods based on the detected optical sensor signal and to execute the baking program based on this determined shape.In this process, geometric parameters such as diameter, height and / or edge length are preferably also determined.

[0003] DE 10 2019 107 859 A1 discloses a method for operating a cooking appliance with a cooking area and a processing unit for preparing food. The food is monitored in the cooking area during the cooking process. A camera device is used to repeatedly determine the size of the food during the cooking process. From this, a measure of the change in size of the food over time is calculated. Depending on this measure of change in size, the food is assigned to at least one food group.From the area of ​​the image elements assigned to the food being cooked, at least an approximate base area of ​​the food can be determined. From the difference in the distances between the image elements assigned to the food and those adjacent but unassigned image elements, at least an approximate height of the food can be determined. From the base area and the height, at least an approximate volume of the food can be determined, and a change in volume over time can be used as a measure of the change in size of the food. A time-dependent progression of an estimated value of the food volume can be continuously documented at the start and during cooking. Electronics can continuously analyze the time-dependent progression of the estimated food volume in relation to the selected cooking program and intervene in the control of the cooking program. This can be done, for example, by...The cooking process can be adjusted by changing the cooking chamber temperature, providing assistance through dialogue with the user towards the end of the cooking time, and automatically ending the cooking process. Based on this analysis and information, the electronics then intervene in the cooking process, for example by switching off the heating and other energy supply, and if necessary by keeping the food warm or (rapidly) cooling it.

[0004] DE 10 2019 107 812 A1 discloses a method for operating a cooking appliance with a cooking chamber for preparing food, wherein the food in the cooking chamber is monitored during the cooking process and wherein images of the cooking chamber are captured over time by means of a camera device, the images each consisting of a multitude of image elements and being evaluated by means of a processing device. Image elements that change over time are identified and assigned to the food in order to distinguish them from image elements originating from outside the food. This method exploits the fact that image elements containing food behave completely differently from image elements containing the cooking chamber with cooking chamber walls, containers, etc. This difference relates not only to changes caused by browning and discoloration, but also, for example, to...This also applies to distances caused by changes in volume or surface temperature. The images captured by the camera device can contain spatial image information from the cooking area, whereby at least one temporal change in the spatial image information, preferably a change in distance to the camera device, is evaluated to identify the image elements that change over time.

[0005] DE 10 2019 212 364 A1 discloses a method for operating a household cooking appliance with a cooking chamber and at least one camera which is equipped to take pixel-based images from the cooking chamber, wherein (a) at least one image from the cooking chamber is taken by means of the at least one camera and (b) the image is evaluated excluding the brightness values ​​of its associated pixels.

[0006] It is the Aufgabe The present invention aims to overcome the disadvantages of the prior art, at least in part, and in particular to provide a simple and robust way to calculate a quantity relating to the volume of heat-treated food from a sequence of images.

[0007] This problem is solved according to the features of the independent claims. Preferred embodiments can be found in particular in the dependent claims.

[0008] The problem is solved by a method for determining the volume of food being cooked in a treatment chamber of a household cooking appliance, in which Images are taken from the treatment room in chronological sequence; in one, especially the first, image of the sequence, image points belonging to the food being cooked are identified; for subsequent images, a direction and speed of movement of the image points are calculated using the method of optical flow, in particular the method of dense optical flow; the directions of movement of the previously identified image points are classified into classes for different directions of movement; the number of image points falling into the respective classes is counted; at least one quantity relating to the volume of the food being cooked is calculated from the numbers; and then, if the at least one calculated quantity fulfills at least one predefined criterion, at least one operating parameter of the household cooking appliance influencing the treatment of the food is varied.

[0009] This method offers the advantage that the size relating to the volume of the heat-treated food can be reliably determined regardless of the size of the food, its positioning, and the type of container used.

[0010] A household cooking appliance can be, for example, an oven, a steamer, a microwave, a food processor, or any combination thereof, such as an oven with steaming functionality, particularly steam cooking functionality. The treatment chamber can also be called a cooking chamber, even if the food is not actually cooked in it, but rather treated, for example, by being moistened at room temperature.

[0011] This method is particularly advantageous if the food undergoes a noticeable change in volume, at least during part of the treatment. It is a further development that the food being cooked is a dough product, e.g., bread dough, etc.

[0012] The timing sequence can, for example, include taking pictures at intervals of several seconds to minutes, such as every 5 s, 10 s, 30 s, 1 min, 2 min, etc.

[0013] The fact that pictures are taken from the treatment room includes, in particular, that food being cooked in the treatment room is also depicted in the pictures.

[0014] It is a further development that at least some of the images are captured by a camera integrated into the household cooking appliance. The camera can be positioned, for example, on the ceiling, a side wall, or a rear wall of the cooking chamber (also known as the muffle), or on a door that closes a loading opening of the cooking chamber. It is a further development that the camera is angled into the cooking chamber, which is advantageous for capturing vertical movement of the food being cooked. The camera can also be referred to as a cooking chamber camera. The camera is, in particular, a pixel-based digital camera, specifically a digital color camera.

[0015] The household cooking appliance can have one or more interior cameras. The procedure can be carried out using images from only one camera or using images from multiple cameras.

[0016] Identifying the sequence of pixels belonging to the food being cooked can also be called segmentation. This identification can be carried out, for example, through object recognition, determination of brightness and / or color differences (e.g., dark cooking chamber walls, cooking trays and containers, light-colored food), etc.

[0017] The single image in which the pixels are identified can, in particular, be the first image of the image sequence used to carry out the procedure.

[0018] The optical flow method, and in particular the dense optical flow method, is fundamentally well-known. It captures the motion of objects in images by tracking the image points characteristic of the objects across a sequence of images, specifically from image to image. For successive images, each image point in the image plane exhibits a direction of motion (represented by the polar angle φ) and a velocity (represented by the radius r), both expressed in polar coordinates. This corresponds to assigning a vector lying in the image plane to a respective image point, where the vector's direction corresponds to the image point's direction of motion and its length to its velocity. This motion can be continued across more than two images.In particular, this allows the progression of the movement of the image points in the image plane to be tracked for multiple images.

[0019] The fact that the movements of the previously identified ("segmented") pixels are classified into classes for different directions of movement includes, in particular, that only the movements of the previously identified pixels are classified into classes for different directions of movement.

[0020] It is a further development that the optical flow method is applied only to the image points previously identified as belonging to the food being cooked. It is a particularly advantageous further development that the optical flow method is applied to all image points, and subsequently only those identified as belonging to the food being cooked are considered. It is a further development that the identified image points for all images in the image sequence used correspond to the image points identified in the first image.

[0021] Classifying the movements of pixels for different directions of movement involves, in particular, checking whether the directions of movement determined by the optical flow method are related to at least one predefined direction of movement. If so, the corresponding pixel is assigned to a class associated with that direction of movement; otherwise, it is not. It is therefore possible that none of the previously identified pixels will be assigned to at least one of the classes, but will remain "classless" – for the entire treatment process or for certain segments thereof.

[0022] It is a further development that the pixels are classified unweighted, meaning that each pixel has the same unit value, e.g., "1". The number of a class therefore corresponds exactly to the number of pixels in that class.

[0023] It is a further development that the pixels are classified without weighting, meaning that each pixel is assigned a weighting factor, thus assigning each pixel a "weighted number". Consequently, different pixels can contribute to the (total) number in the classes to varying degrees. It is a further development that the weighting depends on the movement speed of the respective pixel. For example, the weighting factor, and therefore the weighted number, can be larger the greater the movement speed. In particular, the weighting factor can be a function of the movement speed, especially proportional to the movement speed.

[0024] It is a further development that the different directions of movement include movements in opposite directions along the same axis. It is a further development that the different directions of movement include movements in opposite directions along multiple axes.

[0025] The calculation of at least one quantity relating to the volume of the food being cooked from the numbers includes, in particular, the derivation or estimation of this quantity from the absolute value and / or the relation(s) between numbers of at least two classes. The relationship can also include the number of classless pixels. For example, the relationship can be a relation between numbers of different classes and / or classless pixels, a relation to a threshold value, etc. The relationship between the numbers and the at least one quantity can be determined, for example, through experiments, simulations, and / or machine learning, etc. The relationship can be stored in the household cooking appliance, for example, in the form of a table or a function with the numbers as variables.

[0026] Fulfilling at least one predefined criterion can involve fulfilling a condition or relation, fulfilling one condition from a group of several conditions, and / or simultaneously fulfilling at least two conditions from a group of several conditions. The criterion can, for example, include reaching, exceeding, and / or falling below at least one predefined threshold.

[0027] It is a further development that the criterion includes fulfilling at least one relationship between the numbers assigned to the respective classes and, if applicable, the number of classless pixels. In this further development, no physical quantity relating to volume is explicitly calculated; rather, the at least one quantity relating to volume is given implicitly or analogously through the relationship between the numbers.

[0028] This configuration includes both movement in the direction of gravity and movement against it, i.e., movements in opposite directions along the axis of gravity. This offers the advantage of a particularly simple way to check whether the food being cooked is moving upwards (hereinafter also referred to as "lifting") or downwards (hereinafter also referred to as "lowering"). Furthermore, this method allows for a particularly straightforward quantitative determination of the lifting and lowering. The axis of gravity corresponds very closely to the vertical axis of space, especially the downward-pointing axis. The axis of gravity can be determined in a camera image by its geometric arrangement and orientation, regardless of whether it is loaded with food.

[0029] One implementation assigns a pixel to a first class (corresponding to elevation) if it moves within a first predefined angular deviation from the axis of gravity against the direction of gravity, and to a second class (corresponding to depression) if it moves within a second predefined angular deviation from the axis of gravity in the direction of gravity. This achieves the advantage of providing a particularly simple classification that delivers reliable results.

[0030] The fact that a pixel moves within a specified angular deviation relative to a particular reference axis includes, in particular, that the pixel's direction of movement in the image plane exhibits an angular deviation relative to the reference axis that is neither greater nor less than the specified angular deviation. The angular deviation can be symmetrical or asymmetrical with respect to the axis; that is, the magnitude of the angular deviation can be the same or different in a clockwise direction and counterclockwise direction.

[0031] It is a further development that a pixel is classified into the first class if the angular deviation to the downward-directed gravitational axis is in the angular ranges [φ 1l ; φ 1u ] or ]φ 1l ; φ 1u [ , where the boundary conditions 90° < φ 1l ≤ 180° and 180° ≥ φ 1u < 270° apply.

[0032] For a reliable and robust determination of the volume-related quantity, it is particularly advantageous to choose 150° ≤ φ 1l ≤ 160° and / or 200° ≤ φ 1u ≤ 210°. In general, 180° - φ 1l = φ 1u - 180° (symmetrical angular deviation) or 180° - φ 1l <> φ 1u - 180° (asymmetrical angular deviation) can apply.

[0033] It is a further development that a pixel is classified into the second class if the angular deviation to the downward-directed gravitational axis is in the angular ranges [φ 2l ; φ 2u ] or ]φ 2l ; φ 2u [ , where the boundary conditions 270° < φ 2l ≤ 360° ≡ 0° and 0° ≥ φ 2u > 90° apply.

[0034] For a reliable and robust determination of the volume-related quantity, it is particularly advantageous to choose 330° ≤ φ 2l ≤ 340° and / or 20° ≥ φ 2u > 30°. In general, 360° - φ 2l = φ 2u (symmetrical angular deviation) or 360° - φ 2l <> φ 2u (asymmetrical angular deviation) can apply.

[0035] For example, if the reference axis is the downward-pointing gravity axis and the first and second angular deviations are symmetrical, e.g., with a magnitude of 20°, those pixels whose directions of movement lie within an angular range of [-20°; +20°] or ]-20°; +20°[ to the gravity axis are classified into the first class, and those pixels whose directions of movement lie within an angular range of [160°; 200°] or ]160°; 200°[ to the gravity axis are classified into the second class.

[0036] This configuration includes different directions of movement such as a movement away from the surface center of gravity of the food being cooked and a movement towards the surface center of gravity of the food being cooked. This offers the advantage that, in addition to or as an alternative to lifting and lowering, a lateral expansion of the food being cooked (hereinafter referred to as "expansion") and a lateral contraction of the food being cooked (hereinafter referred to as "contraction") can also be taken into account in order to calculate at least one dimension of the food being cooked that relates to its volume.

[0037] It is a further development that the surface centroid corresponds to the geometric centroid of the segmented image point in the image plane.

[0038] One embodiment involves determining the surface center of gravity of the food being cooked. For each image point belonging to that food, a connecting line to the surface center is defined. An image point is assigned to a third class (corresponding to expansion) if it moves away from the surface center of gravity within a third predefined angular deviation relative to a connecting line. An image point is assigned to a fourth class (corresponding to contraction) if it moves towards the surface center of gravity within a fourth predefined angular deviation relative to a connecting line. This provides the advantage of a reliable measure for the expansion and contraction of the food being cooked in a particularly simple and easily implemented manner.

[0039] For a pixel to move away from the surface centroid within a predetermined angular deviation from the connecting line, this includes, in particular, that one direction of movement of the pixel in the image plane exhibits an angular deviation from the connecting axis directed from the surface centroid to this pixel, which is neither greater nor less than the predetermined third angular deviation. For a pixel to move towards the surface centroid within a predetermined angular deviation from the connecting line, this includes, in particular, that one direction of movement of the pixel in the image plane exhibits an angular deviation from the connecting axis directed from the surface centroid to this pixel, which is neither greater nor less than the predetermined fourth angular deviation. The angular deviation can be symmetrical or asymmetrical with respect to the connecting axis.

[0040] It is a further development that a pixel is classified into the third class if the angular deviation to the connecting axis directed away from the surface centroid is in the angular ranges [φ 3l ; φ 3u ] or ]φ 3l ; φ 3u [ , where the boundary conditions 270° < φ 3l ≤ 360° ≡ 0° and 0° ≥ φ 3u > 90° apply.

[0041] For a reliable and robust determination of the volume-related quantity, it is particularly advantageous to choose 330° ≤ φ 3l ≤ 360° and / or 0° ≥ φ 3u > 30°. Generally, 360° - φ 3l = φ 3u (symmetrical angular deviation) or 360° - φ 3l <> φ 3u (asymmetrical angular deviation) can apply. Generally, 180° - φ 1l = φ 1u - 180° (symmetrical angular deviation) or 180° - φ 1l <> φ 1u - 180° (asymmetrical angular deviation) can apply. It is a particularly advantageous further development that φ 3l = 360° applies, i.e., that a pixel is only classified into the third class under the condition that the angular deviation of its direction of movement lies within the angular range [0°; φ 3u ], e.g. within the angular range [0°; 30°].

[0042] It is a further development that a pixel is classified into the fourth class if the angular deviation to the connecting axis in a clockwise direction lies in the angular ranges [φ 4l ; φ 4u ] or ]φ 4l ; φ 4u [ , where the boundary conditions 90° < φ 4l ≤ 180° and 180° ≥ φ 4u < 270° apply.

[0043] For a reliable and robust determination of the volume-related quantity, it is particularly advantageous to choose 150° ≤ φ 4l ≤ 160° and / or 180° ≤ φ 4u ≤ 210°. Generally, 180° - φ 1l = φ 1u - 180° (symmetrical angular deviation) or 180° - φ 1l <> φ 1u - 180° (asymmetrical angular deviation) can apply. It is a particularly advantageous refinement that φ 4u = 180°, i.e., that a pixel is classified into the fourth class only if the angular deviation of its direction of motion lies within the angular range [φ 4l ; 180°], e.g., within the angular range [150°; 180°].

[0044] For example, if the third and fourth angular deviations with respect to the connecting axis serving as the reference axis are symmetrical, e.g., specified in magnitude to 25°, those image points whose directions of movement lie within an angular range of [-25°; +25°] or ]25°; +25°[ in the direction of the connecting axis pointing away from the surface centroid would be classified in the third class, and those image points whose directions of movement lie within a range of [160°; 200°] or ]160°; 200°[ to the connecting axis would be classified in the fourth class.

[0045] One implementation involves assigning a pixel to a specific class only if its motion speed exceeds a predefined threshold. This makes the determination of the volume more robust, because pixels to be classified must then exhibit a minimum speed, and consequently, barely moving pixels are disregarded.

[0046] It is a further development that the speed of movement corresponds to the scalar length of the motion vector of the respective pixel.

[0047] It is a further development step to identify the pixels in the image that correspond to the food being cooked, based on their color. This is particularly easy to implement and is based on the understanding that food being cooked typically has a noticeably different color than the usually dark blue, dark gray, or black walls of the cooking chamber, the usually black baking tray, the usually silver wire rack, and the cooking containers. This is especially true for fresh dough, which is typically light brown. Pixels whose color coordinates lie in a subspace of the image's color space specified for the color of the food being cooked are assigned to the food being cooked; other pixels are not.

[0048] It is further information that the recorded image is an RGB image, meaning that the possible colors of the pixels lie in the RGB color space.

[0049] It is a particularly easy-to-implement training method to identify the pixels in an image that correspond to the food being cooked by determining whether their H-values, based on an HSV (Hue, Saturation, Value) color coordinate system, fall within a predefined range. This offers the advantage that the association of a pixel with light brown food can be reliably determined using only one coordinate axis of the color space: its position on the H-axis. This method takes advantage of the fact that brown tones lie on the H-axis in the HSV color space, which is particularly beneficial for brown food being cooked, especially light brown food, and especially dough.

[0050] Consequently, for example, pixels belonging to a dough piece can be identified by their values ​​on the H-axis ("H-values") lying within a range of values ​​defined by a lower threshold ("lower "H-threshold") and an upper threshold ("upper "H-threshold"), which includes the brown tones of typical dough pieces.

[0051] If an image is captured in a color space other than the HSV color space (e.g., as an RGB image), the color coordinates of the pixels can be converted or transformed from the original color space into the HSV color space.

[0052] As an alternative to the HSV color space, an HSL ("Hue, Saturation, Lightness") color space – adapted especially with regard to limit values ​​– can also be used advantageously.

[0053] It is a configuration in which the at least one quantity relating to the volume of the food being cooked includes at least one quantity from the group: volume of the food being cooked, change in the volume of the food being cooked over time and / or rate of change in the volume of the food being cooked.

[0054] This configuration includes at least one predefined criterion: reaching a maximum volume of the food being cooked and / or reaching a minimum change in the volume of the food being cooked. This has the advantage that reaching an extreme value and / or approaching a steady-state volume is particularly easy to identify, especially reaching a maximum volume of the food being cooked and / or reaching a minimum change in volume. The minimum change in volume can be assumed, for example, if the change in volume, especially the increase in volume, of the food being cooked between two successive images is less than a predefined threshold.

[0055] This configuration includes at least one operating parameter influencing the cooking process that comprises at least one parameter from the group consisting of temperature in the treatment chamber (also referred to as "cooking chamber temperature") and / or humidity in the treatment chamber. This offers the particular advantage for foods that are sensitive to changes in volume due to changes in cooking chamber temperature and / or humidity in the treatment chamber, as the cooking process can be adapted to specific food conditions with exceptional reliability and speed, thereby improving the cooking result.

[0056] One variation of the method involves using dough as the cooking material. This method is particularly advantageous for such materials because the volume and / or changes in volume of dough are especially reliable indicators of the presence and / or completion of specific treatment phases. If the treatment phases, and especially their end, can be determined more reliably, the cooking results are also significantly improved.

[0057] It is a further development that the dough is bread dough, because the volume of bread dough, or a quantity derived from it, is particularly easy to measure, and bread dough also has different treatment phases, which show a noticeably different volume behavior and require different environmental conditions for particularly good results.

[0058] In bread baking, the dough first rises significantly in high humidity and high oven temperatures (known as "dough rise"). This is followed by a transition to a second phase (known as "finishing" or "browning"), in which the bread dough largely maintains its volume while developing a browned crust, or even decreases slightly. The second phase typically utilizes lower temperatures and lower humidity in the oven than the first phase and lasts longer. The more precisely the transition point between the two phases can be determined in order to adjust the oven environment (especially with regard to oven temperature, but also with regard to humidity level and / or the heating elements used, etc.), the better the baking result.

[0059] This design incorporates a reduction in temperature and / or humidity in the processing chamber when the dough reaches its maximum volume and / or minimum volume change. This ensures particularly good results during the transition from the rising phase to the browning phase.

[0060] The task can also be solved by a household cooking appliance, featuring a treatment room equipped for heat-treating food placed therein, at least one camera for recording images from the treatment room, and a data processing device equipped to identify image points belonging to the food in a sequence of images, to calculate a direction and speed of movement of the image points for subsequent images using the optical flow method, in particular the dense optical flow method, to classify the movements of the previously identified image points into classes for different directions of movement, to count the number of image points falling into the respective classes, to calculate at least one quantity relating to the volume of the food from these numbers, and then, if the at least one calculated quantity fulfills at least one predetermined criterion,to vary at least one operating parameter of the household cooking appliance that influences the cooking process.

[0061] The household cooking appliance can be trained analogously to the procedure, and vice versa, and has the same advantages.

[0062] The properties, features and advantages of this invention described above, as well as the manner in which they are achieved, will become clearer and more easily understood in connection with the following schematic description of an exemplary embodiment, which will be explained in more detail in conjunction with the drawings. Fig. 1 shows an image taken from a treatment room, depicting the surface of bread dough and showing several coordinate systems; Fig. 2 shows the image from Fig.1 with the center of gravity axis; Fig. 3 shows the image from Fig.1 with the connecting axis between the surface center of gravity and the image point; and Fig. 4 shows a possible procedure for carrying out the procedure.

[0063] Fig.1 Figure 1 shows a color image B taken from a treatment chamber in the form of a cooking chamber 1 of a household cooking appliance 2 using a digital camera 7. The household cooking appliance 2 can be, for example, an oven with additional steam treatment functionality, e.g., so-called "Added Steam". Image B shows a cooking chamber wall 3, a wire rack 4, a rectangular baking pan 5 placed on the wire rack 4, and a dough piece 6, in particular bread dough, placed in the baking pan 5. Image B typically has p pixels in its x-direction or along its x-axis x and q pixels in its y-direction or along its y-axis y, e.g., 640 x 480 pixels, 1024 x 768 pixels, etc. The x-axis x and the y-axis y intersect here at the geometric centroid SP of the pixels belonging to the dough piece 6.

[0064] Also shown are the Cartesian coordinate axes x' and y', which also intersect at the geometric centroid SP. The x' axis corresponds to the downward-pointing centroid axis and can therefore also be designated "g". The centroid axis g is identical for all image points. The centroid axis x' or g results from the arrangement and orientation of the camera 7 and is therefore independent of the presence of the dough piece 6. The (x', y') coordinate system is rotated here relative to the (x, y) coordinate system.

[0065] Also shown are the Cartesian coordinate axes x" and y", which also intersect at the geometric centroid SP, with the x" axis aligned along the longitudinal direction of the dough piece 6. The optional use of this coordinate system is advantageous for easily relating the movement of the image points to the food being cooked.

[0066] The household cooking appliance 2 further comprises a data processing unit 8 with which the camera 7 is data-connected and can control the camera 7 for image acquisition. The data processing unit 8 is also configured, e.g., programmed, to process the images B transmitted by the camera 7, for example, to calculate a quantity relating to the volume of the dough piece 6, such as a change in volume over time. The data processing unit 8 can also be configured to vary at least one operating parameter influencing the treatment of the dough piece 6, such as a cooking chamber temperature, humidity, etc. In particular, the data processing unit 8 can be a central control unit of the household cooking appliance 2.

[0067] Fig.2 The image B shows Fig.1 The center of mass axis g is drawn through an image point BP1 assigned to the dough piece 6. A motion vector v(BP1), calculated from successive images B using the dense optical path method, can be assigned to image point BP1. The motion vector v(BP1) can be expressed as the scalar product of its normalized direction of motion and its length, in particular in polar coordinates r (length) and φ (angle to the center of mass axis g).

[0068] Furthermore, a first angular range [160°; 200°] in the direction of gravity and ± 20° against the direction of gravity (corresponding to the direction opposite the axis of gravity g) is shown. If the motion vector v(BP1) lies within the first angular range, this can be considered as the lifting of the dough piece 6 at the location of this image point BP1.

[0069] Furthermore, a second angular range [340°; 20°] or ± 20° in the direction of gravity (corresponding to the direction along the gravitational axis g) is shown. If the motion vector v(BP1) lies within this second angular range, this can be interpreted as the dough piece 6 sinking at the location of this image point BP1.

[0070] Fig.3 The image B shows Fig.1 with a connecting axis c(BP1) between the surface centroid SP and an image point BP2, and with a connecting axis c(BP2) between the surface centroid SP and an image point BP3 of the dough piece 6. The connecting axes c(BP2) and c(BP3) point from the surface centroid SP to the respective image point BP2 and BP3.

[0071] Furthermore, the motion vectors v(BP2) and v(BP3) of image points BP2 and BP3, respectively, are shown. While the motion vector v(BP2) points away from the surface centroid SP within the third angle range shown, e.g., ± 20° along the connecting axis c(BP2), the motion vector v(BP3) points towards the surface centroid SP within the fourth angle range shown, e.g., ± 20° relative to the connecting axis c(BP3). This can be interpreted as the dough expanding at the location of image point BP2 and contracting at the location of image point BP3.

[0072] It is possible that none, some, or all of the image points associated with dough piece 6 have motion vectors v that lie within the first or second angular range and / or that lie within the third or fourth angular range. It is also possible that none, some, or all of the image points associated with dough piece 6 have motion vectors v that do not lie in any of the angular ranges.

[0073] Fig.4 shows a possible procedure for carrying out the process of baking bread.

[0074] After loading the proofing chamber 1 with the dough piece 6, a bread-baking process is started, controlled, for example, by the data processing unit 8. In a first step S1, a first RGB image B1 of the proofing chamber, including the slatted rack 4, the baking pan 5, and the dough piece 6, is taken using the color digital camera 7, for example, controlled by the data processing unit 8, analogous to image B from Fig.1 .

[0075] In step S2, the RGB color coordinates of image B 1 are converted into HSV (Hue, Saturation, Value) color coordinates, e.g. by the data processing unit 8.

[0076] In a third step S3, for example using the data processing unit 8, those pixels are identified whose H-values ​​lie within a value range defined by a lower threshold (lower "H-threshold") and an upper threshold (upper "H-threshold"), which corresponds in particular to the light brown color of a fresh dough piece 6. The result of this "segmentation" is an image mask in which all detected pixels are assigned a specific piece of information, e.g., "1," and all other pixels are assigned a different piece of information, e.g., "0." It is assumed that only the pixels belonging to the dough piece 6 lie within the specified value range. These pixels can also be referred to as "segmented" pixels.

[0077] It is a further development step that this image mask is determined once from the first image B 1 and then remains unchanged for the rest of the process.

[0078] In a fourth step S4, the surface centroid SP of the segmented pixels assigned to the dough piece 6 is determined or calculated, e.g., the geometric centroid, using the data processing device 8.

[0079] In a fifth step S5, e.g. controlled by the data processing unit 8, another image B i with i = 2 is taken.

[0080] In a sixth step S6, the dense optical flow method is applied to images B i and B i-1, for example, using data processing equipment 8. The resulting motion vector v in the (x, y) image plane for each image point is first transformed from the Cartesian coordinate system into a motion vector v of a polar coordinate system, whose length r indicates its velocity and whose polar angle φ indicates its direction of motion. The result is a motion matrix with x · y motion vectors v, each assigned to the image points located at the same matrix position. The motion matrix thus allows for an estimation of the motion of the individual image points.

[0081] In a seventh step S7, the image mask calculated in step S3 is applied to the motion matrix resulting from step S6, for example by means of the data processing unit 8. This is done in particular by scalar multiplication of the corresponding image points, so that only the motion vectors assigned to the image points of dough piece 6 are retained as a result, and no motion vector or a motion vector of length zero is assigned to the remaining image points. In other words, a motion matrix is ​​thus generated in which only the motion vectors assigned to the image points of dough piece 6 are considered.

[0082] In step S8, the segmented pixels are checked, for example using data processing unit 8, to see if their direction of movement lies within a predefined first angular range relative to the axis of gravity g, i.e., whether lifting has occurred at these pixels. If this is the case, the pixel is classified into a first class or assigned to the first class. Step S8 yields the number n 1 of all such pixels.

[0083] In step S9, the segmented pixels are checked, for example using data processing unit 8, to see if their direction of movement lies within a predefined second angular range in the direction of the gravitational axis g, i.e., whether a depression has occurred at these pixels. If this is the case, the pixel is classified into a second class or assigned to the second class. Step S9 yields the number n 2 of all such pixels.

[0084] In step S10, the segmented pixels are checked, for example using data processing unit 8, to see if their direction of movement lies within a predefined third angular range along a connecting axis between the surface centroid SP and the respective pixel, i.e., whether an expansion has taken place at these pixels. If this is the case, the pixel is classified into or assigned to a third class. Step S10 yields the number n 3 of all such pixels.

[0085] In step S11, the segmented pixels are checked, for example using data processing unit 8, to see if their direction of movement lies within a predefined fourth angular range opposite to the connecting axis between the surface centroid SP and the respective pixel, i.e., whether a contraction has occurred at these pixels. If this is the case, the pixel is classified into or assigned to a fourth class. Step S10 yields the number n 4 of all such pixels.

[0086] In step S12, the volume change ΔV between the image acquisitions of images B i-1 and B i is determined from the numbers n 1 and n 2 and / or from the numbers n 3 and n 4, e.g. using the data processing device 8.

[0087] In step S13, it is checked, for example using the data processing unit 8, whether the volume change ΔV fulfills at least one specific criterion, e.g., whether it is less than or less than a predefined positive threshold ΔV thr according to ΔV < ΔV thr, or whether it is less than or equal to ΔV thr according to ΔV ≤ ΔV thr, whether ΔV = 0, or whether ΔV < 0. These criteria correspond to the state of the dough piece 6, in which its volume V hardly increases anymore or even shrinks slightly again, indicating the transition from rising to browning.

[0088] If the volume change ΔV does not meet at least one criterion ("N"), the process branches back to step S5 and another image B i with i := i+1 is acquired. It is particularly advantageous if approximately 10 s elapse between the acquisition of images B i and B i+1.

[0089] If the volume change ΔV meets at least one criterion ("J"), then in step S14, e.g. by means of the data processing device 8, at least one operating parameter of the household cooking appliance influencing the treatment of the dough piece 6 is varied, for example the cooking chamber temperature is reduced, the humidity is decreased, etc., in a further development depending on the type of food being cooked, in particular dough piece 6, for example whether different types of bread dough, croissant dough, etc. are used.

[0090] Rotation matrices can be used to calculate the angular difference between the motion vectors v of the image points and the respective reference axis g or c.

[0091] Of course, the present invention is not limited to the embodiment shown.

[0092] Steps S8 to S11 can be performed in any order.

[0093] The angle ranges can also be fixed or – for example, depending on an operating program used or a known food being cooked – variably adjustable.

[0094] Furthermore, the length |r| of the motion vectors can optionally be taken into account, for example, by classifying image points that have a predefined minimum length |r|min or whose length |rproj| projected onto the respective reference axis (shear force axis g, connecting axis c) has a predefined minimum length |rproj|min. In one advanced training, only image points with a sufficient length are classified. In another advanced training, image points with insufficient length are initially also classified, and these image points are then deleted from the classes at the end of steps S8 to S11.

[0095] Furthermore, the number n 5 of pixels not classified into any of the classes can also be used to determine the volume change ΔV between the image acquisitions of images B i-1 and B i.

[0096] In general, "ein", "eine", etc. can be understood to mean singular or plural, especially in the sense of "at least one" or "one or more", etc., unless this is explicitly excluded, e.g. by the expression "exactly one", etc.

[0097] A numerical specification can also include exactly the specified number as well as a normal tolerance range, unless this is explicitly excluded. B Reference number list

[0098] 1 Cooking chamber 2 Household cooking appliance 3 Cooking chamber wall 4 Wire rack 5 Baking pan 6 Dough piece 7 Camera B Image BP1 Pixel BP2 Pixel BP3 Pixel c Connecting axis SP Center of gravity S1-S14 Process steps v Motion vector xx-axis x'x'-axis x"x"-axis yy-axis y'y'-axis y"y"-axis g Gravity axis

Claims

1. Method (S1 - S14) for determining a volumetric variable of food (6) treated in a treatment chamber (1) of a household cooking appliance (2), in which - images (B) are captured from the treatment chamber (1) in chronological order (S1, S5), - in one image (B) of the image sequence, image points (BP1, BP2, BP3) belonging to the food (6) are identified (S3), - for subsequent images (B), a movement direction and a movement speed of the image points (BP1, BP2, BP3) are calculated (S6) using the optical flow method, in particular the dense optical flow method, - the movement directions of the previously identified image points (BP1, BP2, BP3) are classified (S8 - S11) into classes for different movement directions, - the numbers of image points (BP1, BP2, BP3) that fall into the respective classes are counted (S8 - S11), - from the numbers, at least one variable relating to the volume of the food (6) is calculated (S12), and - then, if the at least one calculated variable satisfies (S13) at least one specified criterion, at least one operating parameter of the household cooking appliance (2) is varied (S14) by the household cooking appliance (2), the operating parameter influencing the treatment of the food (6).

2. Method (S1 - S14) according to claim 1, wherein the different movement directions encompass - a movement in the direction of gravity (g) and - a movement counter to the direction of gravity (g).

3. Method (S1 - S14) according to claim 2, wherein an image point (BP1) - is assigned to a first class when it moves within a first specified angular deviation relative to a gravity axis (g) counter to the direction of gravity (S8) and - is assigned to a second class when it moves within a second specified angular deviation relative to the gravity axis (g) in the direction of gravity (S9).

4. Method (S1 - S14) according to one of the preceding claims, wherein the different movement directions encompass - a movement away from a centre of gravity of the surface (SP) of the food (6) (S10), and - a movement toward the centre of gravity of the surface (SP) of the food (6) (S11).

5. Method (S1 - S14) according to claim 4, wherein a centre of gravity of the surface (SP) of the food (6) is determined and one respective connecting line relative to the centre of gravity of the surface (SP) is determined for the image points (BP1, BP2, BP3) belonging to the food (6), and an image point (BP2, BP3) - is assigned to a third class when it (BP2) moves within a third specified angular deviation relative to a connecting line c(BP2) away from the centre of gravity of the surface (S10), and - is assigned to a fourth class when it (BP3) moves within a fourth specified angular deviation relative to a connecting line c(BP3) toward the centre of gravity of the surface (SP) (S11).

6. Method (S1 - S14) according to one of the preceding claims, wherein an image point (BP1, BP2, BP3) is assigned to a specific class only when its movement speed is above a specified threshold value.

7. Method (S1 - S14) according to one of the preceding claims, wherein the image points (BP1, BP2, BP3) in the image (B) belonging to the food (6) are identified (S3) by the H-values thereof relative to an HSV colour coordinate system being located within a specified value range.

8. Method (S1 - S14) according to one of the preceding claims, wherein the at least one variable relating to the volume of the food (6) encompasses at least one variable from the group: - volume of the food (6), - change in volume of the food (6), - speed of change of the volume of the food (6).

9. Method (S1 - S14) according to one of the preceding claims, wherein the at least one specified criterion encompasses - reaching a maximum volume of the food (6) and / or - reaching a minimum change in the volume of the food (6).

10. Method (S1 - S14) according to one of the preceding claims, wherein the at least one operating parameter influencing the treatment of the food (6) encompasses at least one operating parameter from the group: - temperature in the treatment chamber (1), - humidity in the treatment chamber (1).

11. Method (S1 - S14) according to claims 8 and 9, in which the food (6) is dough, in particular bread dough, and when a maximum volume of the dough and / or a minimum change in the volume of the dough is reached (S13), a temperature and / or a humidity in the treatment chamber (1) is reduced (S14).

12. Household cooking appliance (2) having - a treatment chamber (1) which is designed for the heat treatment of food (6) introduced therein, - at least one camera (7) for capturing images from the treatment chamber (1), and - a data processing facility (8) which is designed - to identify image points (BP1, BP2, BP3) belonging to the food (6) in one image of a sequence of images (B), - to calculate for subsequent images (B) a movement direction and a movement speed of the image points (BP1, BP2, BP3) by means of the optical flow method, in particular the dense optical flow method, - to classify the movement directions of previously identified image points (BP1, BP2, BP3) into classes for different movement directions, - to count the numbers of image points (BP1, BP2, BP3) which fall within the respective classes, - to calculate from the numbers at least one variable relating to the volume of the food (6) and - then, if the at least one calculated variable satisfies at least one specified criterion, to vary at least one operating parameter of the household cooking appliance (2) influencing the treatment of the food (6).