Methods for keeping food in a household refrigeration appliance, as well as household refrigeration appliance

Abstract

Method for storing food in a household refrigeration appliance (1), wherein a food item (15, 19) is detected in a receiving compartment (3) of the household refrigeration appliance (1) by at least one image acquisition unit (11) of the household refrigeration appliance (1), and depending on whether the food item (15, 19) is available in sufficient quantity, wherein at least one separate first mounting device (14) is locally specifically introduced in the receiving compartment (3), by which a locally defined position for a defined first food item is specified for a user, wherein the first food item (15) can be placed on the first mounting device (14) in the receiving compartment (3), and the at least one image which is recorded with the image acquisition unit (11) is evaluated to determine whether the first mounting device (14) has been detected, wherein, if the first mounting device (14) is detected in the image,the quantity of the first food item (15) on it is determined, and wherein the first setup device (14) defines and specifies setup positions and orientations for the first food item (15) so that the first food item (15) is placed on it in a defined order, characterized in that a flexible mat is placed in the receiving space (3) as the first setup device (14), wherein the mat symbolizes the placement of the first food item (15) for a user depending on its size, and the food item is recognized as the first food item (15) depending on its size when evaluating the image.

Description

[0001] The invention relates to a method for storing food in a household refrigerator. A food item is detected in a receiving compartment of the household refrigerator by at least one image acquisition unit of the refrigerator. Based on this detection, it is determined whether the food item is available in sufficient quantity. The invention also relates to a household refrigerator.

[0002] Such a system is known, for example, from DE 11 2016 003 769 T5. This patent discloses, among other things, a refrigeration unit with a camera that can monitor certain food products, such as a carton of milk. When the milk carton is only one-third full, the system can automatically order another carton. The system can also learn response times and automatically adjust an event trigger.

[0003] From DE 10 2016 209 441 A1 a method and a device for recording the contents of a storage compartment of a household refrigeration appliance are known, wherein the recording is carried out with a reading device which can be a camera or an RFID transmitter / receiver.

[0004] US patent 2017 / 0 205 138 A1 discloses a refrigerator with a storage compartment containing multiple bottle shelves, each equipped with a detector to detect the presence of each individual bottle. A display visualizes the detected occupancy of each individual bottle shelf. This "occupancy detection" is supplemented by "bottle detection" using a camera. For this purpose, the operator holds each bottle in front of the camera before placing it in the refrigerator, specifically to visually capture the bottle label.

[0005] WO 2009 / 027 835 A2 discloses an image analysis technique that evaluates the obscuration of easily detectable patterns by objects stored in a storage cabinet. The patterns are applied temporarily or permanently to the bottom of a shelf in the storage cabinet or additionally to the sides or back of a shelf. The patterns can be used to provide information about the products placed on or near them. Furthermore, the patterns can also be used to monitor the removal of individual products. If the patterns are not detected, this indicates that the products have not been completely removed from the storage cabinet.

[0006] One disadvantage of existing systems is that the contents of a food storage compartment can vary considerably. This can depend on the user and meet individual needs. As a result, capturing the contents with a camera that is typically fixed in place on the household refrigerator is difficult, and in some cases, either impossible or not accurate enough.

[0007] The object of the present invention is to provide a method and a household refrigeration appliance in which, or with which, the detection of foodstuffs can be carried out more accurately.

[0008] This problem is solved by a method and a household refrigeration appliance according to the independent claims.

[0009] One aspect of the invention relates to a method for storing food in a household refrigerator. In this method, a food item, if present in a storage compartment of the household refrigerator, is detected by at least one image acquisition unit of the refrigerator. Based on this detection, it is determined whether the food item is present in sufficient quantity in the storage compartment. At least one separate storage device is provided in the storage compartment. A first food item can be placed on this separate first storage device in the storage compartment. This storage device defines a specific placement position for the first food item for the user. The at least one image captured by the image acquisition unit is evaluated to determine whether the first storage device has been imaged.If this is the case and the first display device is detected in the image, the quantity of the first food item on it is determined. This approach thus makes it possible to capture a specific food item with greater accuracy, as the display device defines a spatially predefined area within the recording environment onto which the first food item is to be placed. This allows for improved detection of the first food item and, consequently, a more accurate determination of its quantity.

[0010] In particular, the image processing sequence intelligently searches for this initial placement device during image evaluation, as it essentially triggers the improved detection of the food item to be determined, especially its quantity, within the recording space. The placement device thus represents a physical component that, on the one hand, indicates to the user that they should place their specific food item precisely on this placement device, thus defining its location. This allows the electronic system of the household refrigerator to then accurately determine the quantity of the first food item and decide whether a sufficient quantity is available.On the other hand, the electronic system with the mounting device has a predefined search criterion for image evaluation, which is to be searched for in the image in a first step. This mounting device then defines a limited locality and thus a restricted area within the image, in which the food item on it is clearly identified as the one that is predefined for storage on this mounting device, so that the quantity of this food item can be determined very precisely within the image.

[0011] Advantageously, the image acquisition unit captures an area larger than the area covered by the first display device and any first food items placed on it. During the evaluation of at least one image, the display area is recognized, and only the portion of the image containing this display area is considered for determining the quantity of first food items. This enables faster and more precise identification of the first food items to be assessed, as other areas of the image, which might contain other food items or possibly even another such first food item, are not taken into account.

[0012] The clarity of the statement regarding the quantity of the first food item is thus improved. In particular, the display device predetermines for the user where to place their corresponding food item in order to perform electronic quantity monitoring. If the user then deliberately places such a first food item next to the display device, they do so consciously and are aware that this portion of the first food item, which is not placed on the display device, will not be taken into account in the electronically visualized recording and quantity evaluation of the first food item.

[0013] This approach provides a versatile and flexible system for selecting at least one food item for a user, enabling the automatic electronic determination of its quantity. Furthermore, by focusing solely on the area containing the food item during image analysis, this method filters out other, immediately adjacent food items, thus significantly reducing the risk of misinterpretation. Focusing the analysis on this specific image area also improves the accuracy of the determination and the reliability of the results.

[0014] Preferably, when the first food display is detected in the image, only the quantity of the first food item on it is determined in order to ascertain whether this first food item is present in sufficient quantity in the recording space. Other food items besides the display in the recording space, in particular also first food items that are not on the display, are then not taken into account.

[0015] Preferably, the first setup device can be positioned variably within the recording space, and during image evaluation, the first setup device is recognized regardless of its position if it is captured in the image. This is a further highly advantageous embodiment, as it allows the user to position the first setup device variably and in a variety of ways within the recording space. This enables user-specific positioning of this first setup device, for example, to position other food items in the recording space according to individual user preferences. This might be useful, for instance, to facilitate easy access to these food items and / or to group related food items together within the recording space.Depending on the size and / or quantity, the initial setup device may be positioned in different locations within the recording space. However, by recognizing this in the recording image, these variable positions can be easily accommodated, allowing the setup device to be identified and the quantity of food placed on it to be determined.

[0016] It may also be stipulated that the first mounting device must always be installed in the same position within the recording area to ensure correct camera capture. In such a case, it may also be stipulated that the separate first mounting device in question is permanently installed within the recording area.

[0017] The invention provides that the first display device defines and predefines the placement positions and / or orientations for the first food items on this first display device, so that the first food items are arranged in a defined order. Such predetermination results in a fixed position pattern of the first food items on the display device, thus improving the identification of the first food items, particularly with regard to their quantity on the first display device. Preferably, the placement positions and / or orientations can be defined by shape-specific recesses in the first display device. For example, these recesses can be designed as channels so that elongated first food items can be positioned precisely.Counting the first food items is simplified because there is no unwanted overlap that could be missed during image analysis. The orientation of these indentations can also be determined by their direction. For example, with elongated food items, if these groove-like indentations are oriented in the direction of the refrigerator's depth, the first items can be placed on the rack accordingly and then arranged side-by-side and / or stacked on top of each other, all oriented in the same direction.

[0018] Another example of a recess that defines a positioning position and / or orientation is a square or round indentation. A bottle with a rounded bottom can then be placed in this, or a cardboard box, such as a Tetra Pak, can be securely positioned. Other containers, such as fruit bowls, can also be positioned precisely and securely in this way.

[0019] Preferably, a second food item, different from the first, can be placed on a separate second display unit within the recording space. This second display unit defines a specific position for the second food item for the user. The at least one image captured by the image acquisition unit is evaluated to determine whether the second display unit has been detected. If the second display unit is detected, the quantity of the second food item on it is determined. This is a further highly advantageous embodiment, as it allows both a first display unit and a separate second display unit, optionally located at a distance from the first within the recording space, to be captured practically simultaneously in at least one image.Individual quantity determinations can then be made for each of the different food items assigned and positioned as intended, namely the first food item and the second food item. This allows for a very quick quantity determination process for several different food items, and, in particular, for this process to be carried out independently of each other.

[0020] It may also be provided that at least one display device can be used to place two different food items in a defined manner, and that both food items can be defined and captured by the image capture unit.

[0021] Different food products can generally be distinguished by the fact that the containers in which they are contained differ in shape and / or size. However, a difference can also arise additionally or instead from the color and / or external markings, such as printing, a label, or the like, on a container. In this case, the sizes and / or shapes of the containers may also be the same.

[0022] For example, a container could be a beverage container.

[0023] It may be stipulated that, upon detection of a specific shape and / or size of beverage container, a reorder of precisely that beverage, particularly from a specific manufacturer, is automatically triggered. It may also be stipulated that, upon detection of a specific beverage container containing a beverage from a specific manufacturer, a reorder of a different beverage from that manufacturer is triggered, or a different variety of the detected beverage is ordered from the same manufacturer. It may also be stipulated that, upon detection of a beverage container containing a beverage from a specific manufacturer, a reorder of that beverage variety from a different manufacturer is automatically triggered.

[0024] This is particularly advantageous when the two food items are different but are necessary for preparing a beverage or dish. For example, this might be the case when a mixed drink can be made with these two liquids. In such situations, it is always beneficial to have information on the quantities of both items, allowing for the identification of insufficient quantities of either. This enables an individualized and optimal response. Similarly, with other food items, such as fruit or other ingredients like sauce containers, cream containers, or the like, the quantity can be determined individually, allowing for a decision on whether an additional order is necessary.

[0025] In particular, this information about the quantity of at least one food item can be used to determine whether a notification is issued to a user and / or whether a reorder of this food item is automatically carried out by a machine-based, especially electronic, management system of the household refrigeration appliance, particularly if generated electronically or optically. This allows for an intelligent response to the needs of at least one user of the household refrigeration appliance and enables appropriate action to be taken as required.

[0026] It can be ideally implemented that a user specifies a supplier for a food item, which can be saved by the user, for example. When an automatic reorder is triggered, the electronic management system then contacts the specified supplier to place the reorder. This allows the electronic system to take user-specific preferences into account. For example, very short delivery times can be utilized. Specific special conditions that may be granted to a user can also be applied in this way.

[0027] In an alternative implementation, the electronic management system could search for delivery times and / or prices for the food item to be reordered via a communication network, particularly the internet, and automatically select a supplier based on this information. In such a configuration, once the electronic management system has identified a supplier, this preference could be communicated to the user. Contacting the supplier, specifically to place a reorder, could then only occur after the user confirms this recommendation from the selected supplier. This confirmation could be achieved, for example, through communication via an input device such as a portable communication device like a mobile phone or tablet.Since a user is highly likely to carry such a device, the reordering procedure can be communicated quickly and, if necessary, the user can make the final decision. This also minimizes delays regarding reordering.

[0028] Preferably, a notification is generated indicating that a reorder is required and / or that the relevant food item needs to be transferred from another storage area of ​​the household refrigeration appliance to the first storage unit, optionally to the first storage unit and / or to the second storage unit. It may be provided that a first quantity of the food item is placed on the first storage unit and a further quantity is placed in the storage area, but not on the first storage unit. It may be provided that this storage area cannot be imaged by the image acquisition unit. For example, such an arrangement may be provided for space reasons.It may also be intended that such an arrangement is deliberately made by the user in order to, for example, prioritize the placement of food items that are to be consumed first, with regard to the time of consumption, on the first storage unit, and to have appropriate labeling in place. Furthermore, such an arrangement of the storage area may also be deliberately planned by a user so that, if the quantity of food items on the first storage unit leads to a reorder being suggested, and the user is already aware that a reorder will take a certain amount of time, there will still be sufficient food items available until the reordered quantity arrives.This is particularly advantageous if a quantity threshold, above which a reorder is triggered, is set relatively low or is automatically determined by the system itself.

[0029] In an advantageous embodiment, it may be provided that, for the generation of a notification, it is checked whether the quantity of the detected food has fallen below a first characterization threshold and / or a second characterization threshold. Additionally or instead, it may be provided that a reorder is placed if the quantity of the detected food has fallen below a first characterization threshold and / or a second characterization threshold. Such characterization thresholds, in particular at least two different characterization thresholds, make the assessment scenario more precise.It is possible to more precisely determine the quantity of food placed on a display device that is removed within a given timeframe and, if applicable, what the future demand for that food might be.

[0030] It is particularly advantageous if a notification and / or reorder only occurs when, starting from a certain point, the quantity of a food item monitored on a display device falls below at least the first and second characterization thresholds. This allows for more precise tracking of the food item's removal interval and thus also enables determination of when, for example, a reorder should be placed and in what quantity. This design is also particularly advantageous for increasing the accuracy of certain food items, especially those whose shape makes them more difficult to visualize and measure in terms of quantity, and for minimizing misinterpretations in the event of a slight fluctuation in quantity around one of the characterization thresholds.In particular, these two characterization thresholds are set with values ​​that differ from each other in such a way that there is a sufficient quantity between them to be able to accurately capture the individual quantity of food taken from the associated display device and then to accurately detect when both characterization thresholds are undercut, in order to then determine the exact quantity of food taken.

[0031] In particular, a further notification and / or a reorder will only be generated if, after an initial notification and / or a reorder, both characterization thresholds have been exceeded again and then these two characterization thresholds have been undercut again.

[0032] The invention provides that a flexible mat is placed in the storage compartment as a first mounting device. This mat is designed to be relatively flat, so that the storage space above the mat is not unduly restricted. Particularly when such a mounting device is placed, for example, on a shelf in the storage compartment of a household refrigerator, sufficient volume remains above the mat to accommodate a larger quantity of food. Furthermore, such a flexible mat is particularly well-suited to the ambient conditions in the storage compartment and is robust against temperature and / or humidity levels. In addition, such a mat can be easily handled by a user, making both the removal and loading of food simple.Furthermore, due to its design, such a mat can be easily recognized in images, thus enabling simple and accurate identification. Additionally, the flat structure of the mat allows for more precise identification of food items placed on it, even when stacked. Frontal detection is particularly advantageous in this case, as the food items are exposed at the front.

[0033] According to the invention, it is provided that the mat, depending on its size and optionally also depending on its color, symbolizes the placement of the first food item for a user, so that when looking into the recording room, the user can very quickly and intuitively recognize which food item he can place at which position in the recording room, namely on this specific display device, in order to then be easily recognized by the electronic management system.

[0034] The invention also provides that, depending on the size and optionally also depending on the color of this mat, the food item is recognized as the first food item when evaluating the image.

[0035] In an advantageous implementation, the electronic management system stores information about which food item is to be placed on the first display stand. This allows the system to automatically recognize that the first food item is present when the first display stand is detected during image analysis and it is determined that food items are present. In this case, actual identification through image analysis as the first food item is no longer strictly necessary. It is then presumed that these are indeed the first food items.

[0036] In an advantageous embodiment, it can be provided that such an assignment, stored in the electronic management system, can be individually modified by a user. Thus, a display device can be used in a user-specific manner, such that, depending on their settings and storage in the electronic management system, different food products are placed on and recognized on this initial display device in different situations.

[0037] The same points made above regarding the first installation device also apply advantageously to at least one second installation device.

[0038] The mounting device, particularly if it is designed as a mat, may be provided with a non-slip surface on the underside and / or the top. This helps to prevent unwanted slippage of the entire mounting device within the receiving area. It also helps to prevent unwanted slippage of food items placed on the mounting device.

[0039] Another aspect of the invention relates to a household refrigeration appliance with a housing in which at least one receiving compartment for food is formed. The household refrigeration appliance has at least one detection unit, in particular an image detection unit, for capturing, especially image-based, at least a partial area of ​​the receiving compartment. The household refrigeration appliance also has at least one mounting device for a food item. Furthermore, the household refrigeration appliance has an electronic management system. The household refrigeration appliance is preferably configured to carry out a method according to the aspects mentioned above or an advantageous embodiment thereof. In particular, this method is carried out using the household refrigeration appliance.

[0040] A household refrigeration appliance may also have additional storage space, such as a shelf, door rack, or drawer. These are separate physical components from the mounting device. A mounting device, for example, can be attached to a shelf. An image capture unit could be, for example, a camera. A household refrigeration appliance can be a refrigerator, a freezer, or a combination refrigerator-freezer. It is designed for storing and preserving food. A recording compartment can be a refrigerator compartment or a freezer compartment.

[0041] In a further advantageous embodiment, the electronic management system of the household refrigerator is designed as a machine learning system. This means that learning effects regarding food storage, the generation of notifications, and / or reordering are evaluated and taken into account for future scenarios. For example, such a machine learning process can be carried out using so-called "deep learning." A convolutional neural network (CNN) can be used as the algorithm. The data acquisition unit can be an image capture unit or, for example, an RFID (radio-frequency identification) unit.

[0042] Advantageously, the electronic management system can automatically adjust the first and / or second characterization thresholds. This adjustment can be based, for example, on the food product being placed on and recorded on a display unit. Alternatively, or in addition, this adjustment can also be based on previously recorded scenarios regarding the loading and unloading of such a food product on the associated display unit.

[0043] In particular, such a physical device provides at least one locally defined, limited, and controllable environment on which a specific foodstuff is to be placed. This allows for improved assessment of the foodstuff with regard to electronic evaluation, especially regarding the detection and determination of the quantity present, particularly when this is based on image capture and analysis. Specifically, this method effectively prevents unwanted overlaps or obscurations by other foodstuffs, which would otherwise limit the accuracy of the analysis.

[0044] To increase detection accuracy and thus reliability, a specific implementation may not require a precise determination of a quantity, such as the exact number of beverage containers like bottles, but instead, depending on the situation, a binary response to the decision of whether a specific threshold, particularly a threshold range, has been breached. These thresholds can be advantageously referred to as characterization thresholds.

[0045] The terms “top”, “bottom”, “front”, “back”, “horizontal”, “vertical”, “depth direction”, “latitude direction”, “height direction” indicate the positions and orientations given when the device is used and arranged as intended.

[0046] Further features of the invention are evident from the claims, the figures, and the description of the figures. The features and combinations of features mentioned above in the description, as well as those subsequently mentioned in the description of the figures and / or shown in the figures alone, are not only usable in the combinations specified, but also in other combinations or on their own, without departing from the scope of the invention. Thus, embodiments of the invention are also to be considered as encompassed and disclosed that are not explicitly shown and explained in the figures, but which can be derived and generated from the explained embodiments by separate combinations of features. Embodiments and combinations of features are also to be considered disclosed that do not exhibit all the features of an originally formulated independent claim.

[0047] Exemplary embodiments of the invention are explained in more detail below with reference to schematic drawings. These show: Fig. 1 a schematic representation of an embodiment of a household refrigeration appliance according to the invention; and Fig. 2 a simplified schematic diagram with regard to a quantity of food that varies over time on a mounting device of the household refrigeration appliance.

[0048] In the figures, identical or functionally equivalent elements are given the same reference symbols.

[0049] In Fig. Figure 1 shows a schematic representation of a household refrigeration appliance 1, designed for storing and preserving food. The household refrigeration appliance 1 can be a refrigerator, a freezer, or a refrigerator-freezer combination.

[0050] The household refrigeration appliance 1 has a housing 2. The housing 2 contains at least one storage compartment 3 for food. The storage compartment 3 can, in particular, be a refrigerated compartment. The storage compartment 3 is bounded by the walls of an inner container 4. The inner container 4 has a loading opening on its front side in the depth direction (z-direction). This opening can be closed by a door 5 of the household refrigeration appliance 1. The door 5 is pivotally mounted on the housing 2.

[0051] In Fig. Figure 1 shows door 5 in the open position. It may be provided that, on an inner side 6 of door 5, which faces the receiving space 3 when door 5 is closed, at least one door shelf 7 is arranged. The door shelf 7 is designed to hold food items. For example, it is designed to hold a beverage container 7a, such as a bottle.

[0052] This door shelf 7 is therefore located on the door side.

[0053] Furthermore, the household refrigeration appliance 1 has a shelf 8 arranged on the housing side within the receiving space 3. In the exemplary embodiment, a first shelf 8 and a second shelf 9 are arranged on the housing side within the receiving space 3. These plate-like shelves are arranged at different heights in the vertical direction (y-direction). Additionally, by way of example, a further food receiving container 10 can also be provided, which here is a tray or a drawer that can be inserted into and removed from the receiving space 3 in the depth direction.

[0054] The household refrigeration appliance 1 also has at least one detection unit 11. This detection unit 11 is, in particular, an image detection unit, such as a camera. The position and number of the detection unit 11 are specified in Fig. Figure 1 is to be understood as merely an example. In particular, this detection unit 11 can be arranged on the door 5, especially on the inside 6, so that the recording area 3 can be detected when the door 5 is closed. In particular, this makes it possible for the detection area of ​​the detection unit 11, which is designed especially as an image detection unit, to be oriented in such a way that, for example, at least that part of the recording area 3 can be detected which extends from the very top down to the lower shelf 9.

[0055] It may be provided that the camera is equipped with a fisheye lens in order to be able to take such a wide-angle shot.

[0056] The household refrigeration appliance 1 preferably also includes a control unit 12. The control unit 12 is, in particular, a component of an electronic management system 13 of the household refrigeration appliance 1. The at least one recording unit 11 is also, in particular, a component of this electronic management unit or this electronic management system 13. The electronic management system 13 can, for example, also be located, at least partially, outside the household refrigeration appliance 1, for example, in a cloud.

[0057] It can be provided that the information from the acquisition unit 11 is transferred to the control unit 12, where it is processed and evaluated. In one embodiment, functions of the control unit 12 can also be outsourced to a cloud.

[0058] In the illustrated embodiment, the household refrigerator 1 has a separate, physical first mounting device 14. This first mounting device 14 is a separate physical component from the door shelf 7 and the shelves 8 and 9. In this embodiment, the first mounting device 14 is designed as a mat. It is positioned on the top of the shelf 8 and, when viewed from the front, on the left side of the receiving space 3. In particular, the first mounting device 14 is arranged on the shelf 8 in a way that allows it to be removed without damage. This enables the user to position it in various other ways within the receiving space 3.As can be seen here, the first support device 14 is arranged in the front area of ​​the shelf 8 when viewed in the depth direction, in particular such that no other objects can be arranged in front of this first support device 14 when viewed in the depth direction. This enables the detection of the first food items placed on the first support device 14 by the detection unit 11 to be particularly accurate when the door 5 is closed.

[0059] The setup device 14 is designed such that a specific first food item is to be placed on it, in particular, for example, by means of a predefined assignment. Such an assignment can also be user-specific. For example, this can be done by corresponding programming or storage. Such an assignment is then stored in the electronic management system 13, in particular in the control unit 12. It can also be advantageously changed by the user.

[0060] The first setup device 14 visually indicates to the user, by means of a physical component, where to position a corresponding first food item in the receiving area 3 so that it can then be specifically detected and evaluated by the detection unit 11. In the illustrated embodiment, the first food items are beverage containers 15, in particular bottles, mounted on the first setup device 14. The first setup device 14 also preferably predefines the respective position and / or orientation of such a first food item 15. In the illustrated embodiment, these beverage containers 15 are arranged side by side, stacked, and positioned horizontally. They are preferably mounted with their bottoms facing the loading opening.This allows the detection unit 11 to detect the quantity of the first foodstuffs on the first installation device 14 in a particularly advantageous and very precise manner. Specifically, it is provided that, depending on this information from the detection unit 11, this quantity determination is carried out by means of the mechanical, and in particular electronic, management system 13, especially the control unit 12. Preferably, it is also determined electronically, depending on this, whether a sufficient quantity of these first foodstuffs is present on the first installation device 14. Such a decision as to whether a sufficient quantity of first foodstuffs is present in the household refrigeration appliance 1, in particular on the first installation device 14, is made by a corresponding comparison with at least one characterization threshold.

[0061] Advantageously, the electronic management system 13 is configured such that the at least one image, which was captured by the acquisition unit 11, which is preferably an image acquisition unit, is evaluated to determine whether the first display unit 14 is recognized in this image. In a further image evaluation step, depending on whether the first display unit 14 is recognized, the quantity of first foodstuffs present on this first display unit 14 is then determined by evaluating the image.

[0062] In particular, as already explained above in the context of an advantageous embodiment, at least one image captured by the image acquisition unit is such that the image captures not only the area of ​​the recording space 3 containing the first mounting device 14, but also a significantly larger area of ​​the household refrigeration appliance 1. Using such an image, the image processing algorithm first searches for the first mounting device 14 within the image. If it is detected, further evaluation considers only that portion of the entire image which shows the first mounting device and any food items placed on it.Therefore, only the image section that forms this first mounting device 14 and, in particular, an area formed on this first mounting device 14 extending upwards and backwards, is considered for further evaluation. Such a coverage area 16 is shown in . Fig. 1 shown as an example.

[0063] Such an image analysis procedure makes it possible to determine a specific quantity of a first foodstuff, which is arranged in a specific location within recording space 3, with particular precision.

[0064] Even if, for example, in recording space 3, other items, particularly foodstuffs, 17, as well as, for example, another first foodstuff 15, are placed next to the first display unit 14 in addition to the first display unit 14, this is specifically not taken into account. The first display unit 14 defines a very specific area in recording space 3, onto which the first foodstuff, if any, placed on it can then be specifically detected and evaluated via image analysis. Thus, this system, and in particular the first display unit 14, is intended to create a situation in which electronic detection and evaluation specifically assess only the quantity of the corresponding foodstuffs that are placed locally on this first display unit 14.

[0065] In the advantageous embodiment already mentioned above, in which the first mounting device 14 is preferably arranged in the receiving space 3 in a way that allows for non-destructive removal and can be positioned at different positions in the receiving space 3, the design of the at least one detection unit 11 nevertheless allows, particularly when an image is taken, the first mounting device 14 to be recognized in the image regardless of its position, and the loading with first food items can be further evaluated locally depending on this.

[0066] In particular, the first mounting device 14 is recognizable due to its geometry and / or size and / or color, especially during image evaluation.

[0067] In an advantageous embodiment, the household refrigeration appliance 1 may have a second, separate mounting device 18, distinct from the first mounting device 14. This second device may, for example, be a mat. In the example in (which is not exhaustive) Fig. In Figure 1, this second support device 18 is arranged, for example, on the upper side of the additional shelf 9. Specifically, when viewed from the front, it is located on the left side and, when viewed in the depth direction, is positioned in the front area of ​​the shelf 9. It may be intended that a second food item 19 is to be placed on the second support device 18. This can also be done, as already explained for the first support device 14, by means of a user-defined setting and stored in the electronic management system 13. In the example shown, the second food item can also be a beverage container, but here not a bottle with its specific shape, but rather an angular, in particular cuboid, package.Here too, the design of the second mounting device 18, in particular specific recesses in its upper surface, defines the positioning positions and / or orientations for these second food items 19. Thus, as with the first mounting device 14, which can also have corresponding recesses, a highly defined and structured positioning pattern can be established for the food items 15 and 19 on the mounting devices 14 and 18. This further improves the detection by the detection unit 11 and allows for greater precision.

[0068] Here too, as with the first mounting device 14, it is provided that the size of the mounting device 14, 18 is significantly smaller than the area of ​​a shelf 8 and 9.

[0069] The arrangement of further food items 20 shown on shelf 9 is also only to be understood as an example, next to the second mounting device 18.

[0070] It can be provided that the single detection unit 11 can simultaneously detect the upper area where the first shelf 8 and the first mounting device 14 are located, and simultaneously detect the area where the second shelf 9 and the second mounting device 18 are located. This can be achieved, for example, with a single image. It can also be provided that the domestic refrigeration appliance 1 has at least two separate detection units 11, in particular two image detection units, the first of which can detect the upper area of ​​the detection space 3 and the other the lower area of ​​the detection space 3, so that the detection space 3 can be completely detected, in particular by these two detection units.

[0071] Here too, the electronic management system 13 can determine, depending on the information, the respective quantity of first foodstuffs 15 and the respective quantity of second foodstuffs 19, which are arranged locally only on the display devices 14 and 18.

[0072] Depending on a specific quantity of a foodstuff 15 and / or 19, the electronic management system 13 can then advantageously issue a notification to a user and / or automatically reorder this foodstuff. The household refrigeration appliance 1, in particular the electronic management system 13, can, for example, communicate with a supplier 22 of such a foodstuff via a communication network 21, in particular via the Internet.

[0073] In this context, the electronic management system 13 can, for example, query and evaluate delivery times and / or prices for a food item that needs to be reordered. Depending on this information, it can then select a specific supplier and initiate the corresponding reorder. This can also be done automatically, allowing the management system 13 to train itself.

[0074] Advantageously, the electronic management system 13 is designed to be self-learning. This means that machine training can be performed to determine when a corresponding notification and / or a reorder of a first food item 15 and / or a second food item 19 should be sent. In this context, it is also possible for a specific characterization threshold, which determines whether a notification should be sent to a user and / or a reorder of a specific food item should be automatically adjusted.

[0075] It may be provided that, as a notification to a user, information may also be provided, for example, regarding the transfer of a food item from another storage area of ​​the household refrigeration appliance 1, particularly one not detectable by the detection unit 11, to the respective installation device 14, 18. This may be the case, for example, if a user places a first food item 15, as in Fig. Figure 1 shows the food item placed on shelf 8 next to the first support device 14 and / or, for example, in the door compartment 7. The support device 14 and / or 18 also symbolizes to the user where they must position an assigned food item 15, 19 to enable automatic recording via the recording unit 11 and assessment via the electronic management system 13.

[0076] In Fig.Figure 2 shows a simplified diagram illustrating a change in the quantity of a foodstuff 15 on the first storage device 14. Two characterization thresholds, Z2 and Z4, are given as examples, each characterizing a specific quantity of the first foodstuff. For instance, if the first storage device 14 is loaded with the first foodstuff 15 at time t1, and an orderly arrangement with a quantity Z5 is carried out, the quantity of the first foodstuff 15 on the first storage device 14 can be monitored from time t1 onwards. As can be seen, the quantity Z5 is greater than the quantities specified by the characterization thresholds Z2 and Z4.

[0077] If, for example, a specific quantity of the first foodstuff 15 is subsequently removed from the first storage device 14 at time t2, and this quantity reaches, for example, the higher second characterization threshold Z4, the electronic management system 13 determines that a sufficient quantity of this first foodstuff 15 is still present on the first storage device 14. If, subsequently, further removal of the first foodstuff 15 from the first storage device 14 occurs at times t3 and t4, it is recognized, for example, that at time t4 the quantity of the first foodstuff 15 on the first storage device 14 has reached the lower characterization threshold Z2.

[0078] If, at a subsequent time t5, another first food item 15 is removed from the first storage device 14, this lower characterization threshold Z2 is undercut and a notification to the user and / or a reorder of the first food item 15 is initiated.

[0079] If, at a further time t6, the first setup device 14 is refilled with first foodstuffs 15 such that at least the lower characterization threshold Z2 is exceeded again, this is also detected by the electronic management system 13.

[0080] In an advantageous embodiment, it can be provided that, if, starting from a filling level of the first setup device 14 greater than both characterization thresholds Z2 to Z4, a notification and / or a reorder is generated when the quantity of first foodstuffs 15 has fallen below both characterization thresholds Z2 and Z4.

[0081] This results in greater accuracy and a more precise statement as to whether, for example, a reorder is actually necessary.

[0082] For example, if the removal and restocking of the first storage device 14 with initial food items 15 is carried out in such a way that the respective quantity fluctuates slightly around a characterization threshold, for example, around characterization threshold Z2, then no notification and / or reorder may be generated. This is because it can then be determined that such a relatively small fluctuation in quantity around this characterization threshold occurs, particularly within a specific time interval, and in this context, a certain degree of error tolerance regarding the recording accuracy may also exist. To avoid an unwanted or unnecessary reorder in this context, such an action is then prevented. Reference symbol list 1 household refrigerator 2 cases 3 Recording room 4 inner containers 5 door 6 Inside 7 door shelves 7a Beverage container 8 first shelf 9 second shelf 10 food storage containers 11 recording unit 12 Control unit 13 electronic administration system 14 first setup device 15 first food 16 Document area 17 more items 18 second setup device 19 Foods 20 food items 21 Communication network 22 providers t1 Time t2 time t3 time t4 time Z2 Characterization Threshold Z4 Characterization Threshold

Claims

[1] A method for storing food in a household refrigeration appliance (1), in which a food item (15, 19) is detected in a receiving compartment (3) of the household refrigeration appliance (1) by at least one image acquisition unit (11) of the household refrigeration appliance (1), and it is determined whether the food item (15, 19) is available in sufficient quantity, wherein at least one separate first mounting device (14) is locally specifically introduced in the receiving compartment (3), by which a locally defined position for a defined first food item is specified for a user, wherein the first food item (15) can be placed on the first mounting device (14) in the receiving compartment (3), and the at least one image which is recorded with the image acquisition unit (11) is evaluated to determine whether the first mounting device (14) has been detected, wherein, if the first mounting device (14) is detected in the image,the quantity of the first foodstuff (15) on it is determined, and wherein the first mounting device (14) defines and specifies mounting positions and mounting orientations for the first foodstuff (15) so that the first foodstuff (15) is applied to it in a defined order, . characterized by , that as the first setup device (14) a flexible mat is placed in the recording space (3), wherein the mat symbolizes the setting up of the first food item (15) for a user depending on its size, and depending on the size, the food item is recognized as the first food item (15) when evaluating the image. [2] Method according to claim 1, characterized by, that the image capture unit (11) captures an area that is larger than the coverage area (16) covered by the first mounting device (14) and any first foodstuffs (14) placed on it, wherein the coverage area (16) is recognized during the evaluation of the at least one image and only the image section of the image with this coverage area (16) is taken into account for determining the quantity of first foodstuffs (15) on the first mounting device (14). [3] Method according to claim 1 or 2, characterized by , that the first mounting device (14) can be positioned variably in the recording space (3), and that the first mounting device (14) is recognized independently of its position during the evaluation of the image. [4] Method according to any one of the preceding claims, characterized by, that the installation positions and / or installation orientations are specified by shape-specific recesses in the first installation device (14). [5] Method according to any one of the preceding claims, characterized by , that a second mounting device (18) is introduced locally in the recording space (3), by which a locally defined mounting position for a second food item is specified for a user, and a second food item (19) different from the first food item (15) can be placed on the second mounting device (18) in the recording space (3), which is separate from the first mounting device (14), and that at least one image, which is recorded with the image acquisition unit (11), is evaluated to determine whether the second mounting device (18) has been detected, and if the second mounting device (18) is detected in the image, the quantity of the second food item (19) on it is determined. [6] Method according to any one of the preceding claims, characterized by , that depending on the quantity of at least one food item (15, 19) recorded, a notification is issued to a user and / or an automatic reorder of this food item (15, 19) is generated by a management system (13) of the household refrigeration appliance (1). [7] Method according to claim 6, characterized by , that a user specifies a supplier for a food item (15, 19), in particular that this supplier is saved by a user, and that the specified supplier is contacted when an automatic reorder is made. [8] Method according to claim 6, characterized by , that the electronic management system (13) searches for delivery times and / or prices for the food to be reordered (15, 19) via a communication network (21), in particular the Internet, and selects a supplier accordingly. [9] Method according to any one of claims 6 to 8, characterized by, that a notification is generated indicating a need to reorder and / or a notification is generated indicating a need to transfer the foodstuff in question (15, 19) from another storage area (7) of the household refrigerating appliance (1), which is not detectable by the image acquisition unit (11), to the installation device (14, 18). [10] Method according to any one of the preceding claims 6 to 9, characterized by, that for the generation of an indication, it is checked whether the quantity of the recorded food (15, 19) has fallen below a first characterization threshold (Z4) and / or a second characterization threshold (Z2) that characterizes the quantity of the food to be assessed (15, 19), and / or a reorder is placed if the quantity of the recorded food (15, 19) has fallen below a first characterization threshold (Z4) and / or a second characterization threshold (Z2) that characterizes the quantity of the food to be assessed (15, 19). [11] Method according to claim 10, characterized by , that a notification and / or a reorder only occurs if, starting from the start of monitoring a quantity of a foodstuff (15, 19) on a display device (14, 18), at least the first and the second characterization threshold (Z4, Z2) are undercut. [12] Method according to any one of the preceding claims, characterized by , that the mat additionally symbolizes the placement of the first food item (15) for a user depending on its color, and depending on the color when evaluating the image, the food item is recognized as the first food item (15). [13] Household refrigeration appliance (1) comprising a housing (2) in which at least one receiving compartment (3) for food is formed, and comprising at least one image acquisition unit (11) for image capture of at least one partial area of ​​the receiving compartment (3), comprising at least one separate mounting device (14, 18) for a food item, and comprising an electronic management system (13), in addition to a shelf (8, 9) or a door shelf (7) or a drawer (10) provided as intended in the household refrigeration appliance (1), wherein the household refrigeration appliance (1) is configured to carry out a method according to one of the preceding claims.

Images