Method and device for detecting the position and / or characteristics of an object inside a device, as well as a household appliance and / or commercial appliance.

Abstract

The presented approach relates to a method (900) for detecting the position and / or characteristics of an object (200) within an interior space (105) of a device (100). The method (900) comprises a step (910) of reading in at least one camera image (220) representing the object (200) within the interior space (105). Furthermore, the method (900) comprises a step (920) of analyzing the camera image (220) to detect the position and / or characteristics of the object (200) within the interior space (105) of the device (100).

Description

[0001] The approach presented here relates to a method and a device for detecting the position and / or nature of an object in the interior of a device, as well as a household appliance and / or a commercial device according to the main claims.

[0002] Modern devices are often already equipped with cameras. These cameras may be positioned in fixed, predetermined locations.

[0003] US2020003484A1 describes a method that uses cameras in devices to detect whether air circulation is obstructed by objects in front of air outlets.

[0004] The approach presented here aims to create an improved method for detecting the position and / or characteristics of an object inside a device, an improved device, and an improved household appliance and / or an improved commercial device incorporating the device.

[0005] According to the approach presented here, this problem is solved by a method for detecting the position and / or characteristics of an object in the interior of a device, by a device, and by a household appliance comprising the device and / or a commercial device comprising the device, with the features of the main claims. Advantageous embodiments and further developments of the presented approach are set forth in the following dependent claims.

[0006] The approach proposed here provides a method for detecting the position and / or characteristics of an object within the interior of a device, in order to solve the aforementioned problem, and the method comprises the following steps: - Reading in at least one camera image representing the object in the interior; and - Analyzing the camera image to determine the position and / or nature of the object inside the device.

[0007] In this context, "device" can specifically refer to a household appliance or a commercial appliance, particularly a refrigeration unit such as a refrigerator or freezer. A camera on the device can be used to capture images of its interior. For this purpose, the camera may include an optical sensor or be designed as an optical sensor.

[0008] In this context, an object can be understood specifically as a food item located inside the appliance, including its packaging. This could, for example, be food in a cooling appliance such as a refrigerator or freezer.

[0009] A camera image can be understood in particular as an image or photograph taken by a camera that is arranged or can be placed in the device.

[0010] Without significant effort or additional costs due to extra hardware, the described approach allows for the derivation of additional information about objects stored in devices with integrated cameras.

[0011] According to one embodiment, in an illumination step, the object can be illuminated using a predefined lighting unit, and in an analysis step, the object's shadow is analyzed to determine its position and / or properties. In addition to the images captured by the cameras inside the refrigerator, shadow images are captured using, for example, differently controlled light sources such as interior lighting. Depending on the angle and selection of the light sources, a multitude of images with varying shadow patterns can be generated. Analysis of these additional shadow images can provide access to supplementary information, particularly regarding depth, obscured objects, surface properties, and / or fill levels within the refrigerator. Changes to food items inside the refrigerator, such as bulging lids, can also be detected.

[0012] According to a further embodiment, the illumination step can be performed using the lighting unit with at least two light sources installed at different locations within the device. The analysis step can include an evaluation of a brightness profile, in particular an umbra and / or penumbra of the object. This offers the advantage that, by activating the light sources installed at different locations, additional information about objects stored in the device can be derived very easily and without significant effort or additional hardware costs using the described approach for devices with integrated cameras. Camera-based identification of food items can thus be facilitated, and additional functions such as fill level determination or food storage in the refrigerator, which may rely on depth information, can be enabled.

[0013] According to one embodiment, the illumination step can be carried out using at least two light sources, wherein the two light sources are arranged on the same inner surface of the device's interior or on different, in particular opposite, inner surfaces of the device's interior. This offers the advantage that objects located behind other objects, for example, can also be detected and thus might not be visible if illuminated from only one side.

[0014] According to a further embodiment, the at least two light sources can be activated at different times during the illumination step. This is advantageous because, for example, mold growth or fermenting dough can be detected over a specific period of time between the activations. During this period, a change in or movement of an object, for example caused by mold growth, can be detected, either alternatively or additionally. This allows, for example, the shelf life of objects in the device, particularly food in a refrigerator, to be better monitored by the user.

[0015] According to one embodiment, an object-free area within the device can be detected during the analysis step. This offers the advantage that the user can better monitor the optimal filling of the device. Specifically, the user can adjust the fill level(s) or the placement of objects within the device according to their needs. If an object-free area is detected within the device, the user can, for example, fill this area with food to better meet their requirements.

[0016] According to a further embodiment, the object's size, surface texture, and / or volume can be detected during the analysis step. This is particularly advantageous because it allows the user to identify and, if necessary, dispose of bloated packaging, provided it contains spoiled or inedible food. It is also conceivable that a fuzzy surface of the object could be detected, which could indicate mold growth.

[0017] According to one embodiment, at least one additional camera image can be acquired during the acquisition step. This additional camera image may have been captured by another camera located at a different position within the device than the camera providing the initial image. This additional camera image may have been acquired at a different time than the initial image, either as an alternative or in addition to the initial image. During the analysis step, the position and / or characteristics of the object can be further determined using this additional camera image. This offers the advantage that, without significant effort or additional hardware costs, the described approach allows for the derivation of supplementary information about objects stored within the device for devices with integrated cameras.The more camera images are imported, the more accurately the user can determine and monitor fill levels or empty levels within the device. By capturing the images at different times, the condition or changes of the objects can also be monitored. Such changes could include, for example, fermenting dough or mold growth on food inside the device.

[0018] According to a further embodiment, during the analysis step, the position and / or nature of the object can be recognized from a three-dimensional image of the object inside the device, derived from the camera image and a subsequent camera image. Using such an image, the user can easily and quickly identify the object within the device and determine which areas should be examined more closely. This can lead to general improvements for subsequent object identification based on image data, i.e., the image. The user can then simply examine the recognized object as an example and / or decide what to do with it. This embodiment can also increase user convenience, for example, by eliminating the need for the user to perform a sometimes complex and / or time-consuming search for (specific) objects.

[0019] According to one embodiment, a time interval can be read in the reading step, which may lie between the capture of the camera image and the subsequent camera image. In the analysis step, the nature of the object can be determined using this time interval, particularly if the time interval is greater than a predefined value. If the time interval is greater than a predefined value, the device can, for example, issue a warning message to the user to inform them that a (specific) object has been stationary inside the device for an extended period. The user can then, for example, check the object for the shelf life of its contents to decide whether or not they still wish to consume the object or its contents. This also increases user convenience.

[0020] According to a further embodiment, a condition of the object can be detected during the analysis step if a position of the object detected from the camera image corresponds within a tolerance range to another position of the object detected from the other camera image. For example, in this case, spoilage of the object's contents can be detected as a condition if the object remains in essentially the same position for an extended period of time. Specifically, the object's position after a certain period can be analyzed and compared with its position at the beginning of that period. By way of example, the object can thus be considered spoiled if it has remained in the same position inside the device for an extended period. In particular, the object can be considered spoiled if the period exceeds a predefined time value.This can be convenient for the user because the device can provide user-friendly, exemplary information about supposedly or potentially spoiled or inedible food items. The user can then simply examine the detected item as an example and / or decide what to do with it. This design can increase user convenience, for example, by eliminating the need for the user to conduct a sometimes complex and / or time-consuming search for (specific) items.

[0021] According to one embodiment, a characteristic of the object can be recognized during the analysis step if a shape, color, and / or surface finish of at least one part of the object, detected from the camera image, deviates by more than a tolerance range from another shape, color, and / or surface finish of at least one further part of the object, which was detected from the further camera image. In particular, the further shape, color, and / or surface finish of the at least one further part of the object can be analyzed after a certain period of time and compared with the shape, color, and / or surface finish of the at least one part of the object at the beginning of that period.For example, an object may be considered spoiled after a specified time period if it has increased in size or changed in color or surface texture, as could be the case with mold growth or fermenting dough. This can be convenient for the user because the device provides user-friendly, example-based information about potentially spoiled or inedible food items. The user can then simply examine the detected object and / or decide what to do with it. This design can increase user convenience, for example, by eliminating the need for a sometimes time-consuming and / or tedious search for specific objects.

[0022] The approach presented here further provides a device designed to execute and / or control the steps of a variant of the method presented here in corresponding units. This embodiment of the presented approach, in the form of a device, also allows the problem underlying the invention to be solved quickly and efficiently.

[0023] The device can be configured to read input signals and, using these input signals, determine and provide output signals. An input signal can, for example, be a sensor signal readable via an input interface of the device. An output signal can be a control signal or a data signal that can be provided at an output interface of the device. The device can be configured to determine the output signals using a processing instruction implemented in hardware or software. For example, the device can include a logic circuit, an integrated circuit, or a software module and may be implemented as, or comprised of, a discrete component.

[0024] A computer program product or computer program with program code that can be stored on a machine-readable medium such as semiconductor memory, hard disk memory, or optical memory is also advantageous. If the program product or program is executed on a computer or device, it can be used to carry out, implement, and / or control the steps of the method according to one of the embodiments described herein.

[0025] According to another embodiment, a household appliance and / or a commercial appliance, in particular a refrigerator, can also be provided that incorporates a variant of the device presented here. The advantages described here can also be realized quickly and efficiently with such an embodiment.

[0026] Although the described approach is based on a household appliance, it can also be used in connection with a commercial or professional device.

[0027] Examples of the approach presented here are shown schematically in the drawings and are described in more detail below. It shows Fig. 1 a perspective view of a device according to an exemplary embodiment; Fig. 2 a schematic representation of a camera, an object and a lighting unit for use in an embodiment; Fig. 3 in a right-hand partial representation a schematic representation of a device according to an embodiment and in a left-hand partial representation a schematic representation of a camera, an object and a lighting unit for use in an embodiment; Fig. 4 a schematic representation of a device and a lighting unit for use in an exemplary embodiment; Fig. 5 in a right-hand partial representation a schematic representation of a device according to an embodiment and in a left-hand partial representation a schematic representation of a three-dimensional image for use in an embodiment; Fig. 6 in a left partial representation a schematic representation of a three-dimensional image for use in an embodiment and in a right partial representation a schematic representation of a lighting unit for use in an embodiment; Fig. 7 in a left partial representation and in a right partial representation each a schematic representation of a lighting unit for use in an embodiment; Fig. 8 in a left-hand partial view, in a middle partial view and in a right-hand partial view, each a schematic representation of a lighting unit for use in an exemplary embodiment; and Fig. 9 a flowchart of an embodiment of the method presented here for detecting a position and / or a characteristic of an object in the interior of a device.

[0028] The same or similar reference symbols are used in the following description for identical or similar elements, and for the sake of clarity, a repeated explanation of the function of these elements is omitted.

[0029] Fig. Figure 1 shows a perspective view of a device 100 according to an exemplary embodiment. A total of three cameras 110, 115, 120 are arranged at different positions in an interior 105 of the device 100. One camera 110 is located in or on a first door of the device 100, another camera 115 is located in or on a second door of the device 100, and an additional camera 120 is located on the top of the device 100.

[0030] Modern refrigerators are often already equipped with cameras. These cameras can be positioned at fixed locations. Images captured by such cameras of the refrigerator's interior often only capture a portion of the interior, specifically only the visible area. Using multiple cameras can improve image capture. However, limitations can arise from camera quality, the resolution of the captured images, and, in particular, food items that obscure the view of areas behind them.

[0031] Conventional cameras and camera lenses can still be quite expensive, so it's unlikely that numerous high-resolution cameras with good lenses will be installed in refrigerators anytime soon. Food items and / or objects that obstruct the camera's view could pose a challenge to adequate analysis of the appliance's interior for some time.

[0032] Consequently, even the use of multiple cameras inside the refrigerator may not guarantee that all food items inside are clearly visible and identifiable in the images at all times and in every position, or that their positions and unambiguous identification are possible. Therefore, determining the fill level of the refrigerator may also only be possible imprecisely from the images.

[0033] Fig. Figure 2 shows a schematic representation of a camera 110, an object 200 (specifically a food item), and a lighting unit 205 according to an exemplary embodiment. The camera 110 is arranged at a fixed and known position in the device. The object 200 is aligned at a fixed position on a turntable. The object 200 rotates about an axis of rotation 210. The lighting unit 205 comprises N fixed light sources 215. The N fixed light sources 215 of the lighting unit 205 are individually switchable or individually switched and are arranged at known positions in the device.

[0034] Camera images 220 of object 200 are captured from different angles “delta_A”. For each angle “delta_A”, each of the N light sources 215 is individually switched or individually switchable. In other words, by a Fig. 3. The illustrated device reads in at least one camera image 220, which represents the object 200 in the interior of the device. A separate camera image 220 is captured for each light source 215.

[0035] The camera image 220 is then analyzed to determine the position and / or characteristics of the object 200 inside the device. According to one embodiment, the characteristics of the object 200 can be determined by identifying its size and / or volume.

[0036] Fig. Alternatively, Figure 2 shows a "Virtual depth perception from 2D to 3D - partially "casting shadows".

[0037] Fig. Figure 3 shows in a right-hand partial representation a schematic representation of a device 100, in particular a cooling device, according to an exemplary embodiment, and in a left-hand partial representation a schematic representation of a camera 110, an object 200 and a lighting unit 205 according to an exemplary embodiment.

[0038] In the right-hand illustration, no turntable is shown, unlike in the left-hand illustration and in Fig. 2 is the case, however, the right-hand part of the illustration shows more cameras and light modules than the left-hand part. These are located at various positions inside the device 105. Fig. Figure 1 shows a total of three cameras 110, 115, and 120 arranged. Camera 110 is located in or on the first door of the device 100, camera 115 is located in or on the second door of the device 100, and camera 120 is located on the top of the device 100.

[0039] The camera 110 shown in the left-hand illustration is the same camera 110 shown in the right-hand illustration at a fixed and known position in the interior 105 of the device 100, or a similar camera. The interior 105 can also be referred to here as the cooling zone and subdivided into Zone 1, Zone 2, Zone 3, and Zone 4. The object 200 in the left-hand illustration is aligned at a fixed position on a turntable illustrated in the left-hand illustration. The object 200 rotates about a rotation axis 210. The lighting unit 205 shown in the left-hand illustration, which comprises N fixed light sources 215, is the same lighting unit 205 shown in the right-hand illustration, or a similar lighting unit.The N fixed light sources 215, also called light modules, of the lighting unit 205 shown in the left and right partial representations are individually switchable or individually switched and are arranged at known positions in the interior 105 of the device 100.

[0040] The in Fig. The additional camera 115 shown, which is located in a different position inside 105 of the device 100 than the camera 110 that provides the camera image 220, records at least one further camera image 320. The further camera image 320, like the camera image 220 also shown here, is made from Fig. 2 from a device 325 – alternatively called cooling unit electronics. According to one embodiment, the further camera image 320 may have been taken at a different time than the camera image 220.

[0041] According to a further embodiment, the additional camera image 320 can be analyzed to determine the position and / or nature of the object 200 within the interior 105 of the device 100. During this analysis of the additional camera image 320, as well as during the analysis of the camera image 220, the size and / or volume of the object 200 can be identified as its nature.

[0042] According to an additional embodiment, during analysis the position and / or nature of object 200 can be recognized from a three-dimensional image of object 200 in the interior 105 of the device 100, obtained from camera image 220 and the further camera image 320. Such a three-dimensional image is shown in the following Fig. 5 and Fig. Figure 6 illustrates this. In other words, using the captured camera images 220, 320 and angles, a rough 3D model can be generated from an exterior surface. According to one embodiment, using the captured camera images 220, 320 and the angles shown in the Fig. The shadows and shadow projections of the N light sources 215 shown in 6 to 8 are added to the rough 3D model.

[0043] Furthermore, according to one embodiment, an object-free area in the interior 105 of the device 100 can be detected during the analysis.

[0044] According to a further embodiment, a time interval can be read in, which lies between the acquisition of camera image 220 and the subsequent camera image 320. Using this time interval, the nature of object 200 can (then) be recognized, particularly if the time interval is greater than a predefined time interval value.

[0045] According to one embodiment, a position of object 200 detected from camera image 220, particularly according to the time interval value, can correspond within a tolerance range to another position of object 200 detected from further camera image 320. According to another embodiment, object 200 can be considered spoiled if it has been in the same position in the interior 105 of the device 100 for an extended period of time, specifically if this extended period is greater than the predefined time interval value.

[0046] According to an additional embodiment, a shape, color and / or surface texture of at least one part of the object 200 detected from the camera image 220, in particular according to the time interval value, can deviate by more than a tolerance range from a further shape, color and / or surface texture of at least one further part of the object 200, which was detected from the further camera image 320.

[0047] As an example only, the object may be considered spoiled after the expiry of the time period if it has increased in size, which could be the case, for example, with mold growth on the object or fermenting dough.

[0048] Fig. Alternatively, figure 3 shows a "Virtual depth perception from 2D to 3D - partially "casting shadows".

[0049] Fig. Figure 4 shows a schematic representation of a device 100 and a lighting unit 205 according to an exemplary embodiment. The N fixed light sources 215, also called light modules, of the lighting unit 205 are individually switchable or individually controlled and are arranged at known positions in the interior 105 of the device 100. According to one exemplary embodiment, a single light source 215 can consist of several individual lights 415 – also referred to as LEDs (light-emitting diodes). According to a further exemplary embodiment, the individual lights 415 can also be individually controllable or controlled.

[0050] At various positions in the interior 105 of the device 100 are the in Fig. 1 and Fig. The three cameras shown are 110, 115, and 120. Camera 110 is located in or on the first door of the device 100, camera 115 is located in or on the second door of the device 100, and camera 120 is located on the top of the device 100.

[0051] Fig. Alternatively, Figure 4 shows a "Virtual depth perception from 2D to 3D - partially "casting shadows"."

[0052] Fig. Figure 5 shows a schematic representation of a device 100 according to an embodiment in the right-hand part and a schematic representation of a three-dimensional device in the left-hand part. Fig. According to an exemplary embodiment. The N fixed light sources 215, also called light modules, of the lighting unit 205 are individually switchable or individually switched and are arranged at known positions in the interior 105 of the device 100.

[0053] At various positions in the interior 105 of the device 100 are the ones in the Fig. 1, Fig. 3 and Fig. The three cameras 110, 115, and 120 are arranged as shown in the illustration. Camera 110 is located in or on the first door of the device 100 in the right-hand part of the illustration, the second camera 115 is located in or on the second door of the device 100, and the additional camera 120 is located on the top of the device 100.

[0054] In the left-hand illustration, objects 200 are arranged in the interior 105 of the device 100 between a first side and a second side of the lighting unit 205. According to one embodiment, these objects could be food items in a refrigerator.

[0055] According to an additional embodiment, the position and / or nature of object 200 can be determined from the in during the analysis. Fig. 2 camera image described and the one in Fig. The three described additional camera images determined three-dimensional Fig. The object 200 inside the interior 105 of the device 100 can be detected. In other words, using the captured camera images and angles, a rough 3D model can be generated from an exterior surface. According to one embodiment, a surface can be added to the rough 3D model using the camera images and the shadows 600 and shadows from the N light sources 215.

[0056] Fig. Alternatively, Figure 5 shows a "Virtual depth perception from 2D to 3D - partially "casting shadows".

[0057] Fig. Figure 6 shows a schematic representation of a three-dimensional structure in the left-hand section. Fig. Figure 1 and Figure 2 show a schematic representation of a lighting unit 205 according to an exemplary embodiment. In both the left and right-hand illustrations, objects 200 are arranged in the interior 105 of the device between a first side and a second side of the lighting unit 205.

[0058] The components located in the device are situated at various positions within the interior (105) of the device. Fig. 1, Fig. 3 and Fig. The illustration shows a total of three cameras, 110, 115, and 120. Camera 110 is located in or on the first door of the device in the right-hand part of the illustration, camera 115 is located in or on the second door of the device, and camera 120 is located on the top of the device.

[0059] According to one embodiment, during illumination using the predefined illumination unit 205, a shadow 600 cast by the objects 200 can be analyzed to determine the position and / or characteristics of the objects 200. According to another embodiment, illumination using the illumination unit 205 can be performed with at least two light sources 215 installed at different locations within the interior 105 of the device. During analysis, a brightness profile and / or a shadow 600, in particular an umbra and / or a penumbra of the object 200, can be evaluated. According to another embodiment, illumination can be performed using at least two light sources 215, wherein the two light sources 215 are arranged on the same inner side of the interior 105 of the device or on different, in particular opposite, inner sides of the interior 105 of the device.According to an additional embodiment, the at least two light sources 215 can be activated at different times.

[0060] According to an additional embodiment, the position and / or nature of object 200 can be determined from the in during the analysis. Fig. 2 camera image described and the one in Fig. The three described additional camera images determined three-dimensional Fig. The object 200 is detected inside the device 105. In other words, using the captured camera images and angles, a rough 3D model can be generated from an exterior surface. According to one embodiment, a surface can be added to the rough 3D model using the camera images and the shadows 600 and the shadows of the N light sources 215.

[0061] Fig. Alternatively, figure 6 shows a “Virtual depth perception from 2D to 3D - partially “casting shadows”.

[0062] Fig. Figure 7 shows a schematic representation of a lighting unit 205 according to an exemplary embodiment, both in a left-hand and a right-hand partial view. In both the left-hand and right-hand partial views, objects 200 are arranged in the interior 105 of the device between a first side and a second side of the lighting unit 205. According to one exemplary embodiment, the light sources 215 of the lighting unit 205 are all switchable.

[0063] According to one embodiment, during illumination using the predefined illumination unit 205, a shadow 600 cast by the objects 200 can be analyzed to determine the position and / or characteristics of the objects 200. According to another embodiment, illumination using the illumination unit 205 can be performed with at least two light sources 215 installed at different locations within the interior 105 of the device. During analysis, a brightness profile and / or a shadow 600, in particular an umbra and / or a penumbra of the object 200, can be evaluated. According to another embodiment, illumination can be performed using at least two light sources 215, wherein the two light sources 215 are arranged on the same inner side of the interior 105 of the device or on different, in particular opposite, inner sides of the interior 105 of the device.According to an additional embodiment, the at least two light sources 215 can be activated at different times.

[0064] Fig. Figure 7 further shows an additional optimization for spatial (3D) image evaluation, in particular the detection of the position of an object 200, the detection of the size and / or volume of the object 200, the detection of a change and / or movement of the object 200 (which may provide an indication of the shelf life of the objects 200, especially food), any free space for objects 200, especially food, a warning in case of impairment of air circulation in the interior 105 of the device and / or the detection of mold growth and / or fermenting dough in the interior 105 of the device.

[0065] Fig. Alternatively, figure 7 shows a “virtual depth perception from 2D to 3D - partially “shadow casting”.

[0066] Fig. Figure 8 shows, in a left-hand, a middle, and a right-hand partial view, a schematic representation of a lighting unit 205 according to an exemplary embodiment. In the left-hand and right-hand partial views, objects 200 are arranged in the interior 105 of the device between a first side and a second side of the lighting unit 205. According to one exemplary embodiment, the light sources 215 of the lighting unit 205 comprise individually switchable LEDs.

[0067] According to one embodiment, during illumination using the predefined illumination unit 205, a shadow 600 cast by the objects 200 can be analyzed to determine the position and / or characteristics of the objects 200. According to another embodiment, illumination using the illumination unit 205 can be carried out with at least two light sources 215 installed at different locations within the interior 105 of the device. During analysis, a brightness profile and / or a shadow 600, in particular an umbra and / or a penumbra of the object 200, can be evaluated. According to another embodiment, illumination can be carried out using at least two light sources 215, wherein the two light sources 215 are arranged on the same inner side of the interior 105 of the device or on different, in particular opposite, inner sides of the interior 105 of the device.According to an additional embodiment, the at least two light sources 215 can be activated at different times.

[0068] Fig. Figure 8 further demonstrates an additional optimization for spatial (3D) image analysis, in particular the recognition of a 3D model for an object, especially for food, the recognition of inflated packaging and / or the shape, color and / or surface texture of the object.

[0069] Fig. Alternatively, figure 8 shows a “virtual depth perception from 2D to 3D - partially “shadow casting”.

[0070] Fig.Figure 9 shows a flowchart of an embodiment of the method 900 presented here for detecting the position and / or characteristics of an object inside a device. Method 900 comprises a step 910 of reading in at least one camera image representing the object inside. Method 900 also comprises a step 920 of analyzing the camera image to detect the position and / or characteristics of the object inside the device.

[0071] In an extended embodiment, the method can include the additional steps “Save image 915” (after step 910), “Loop on step 910” (after step 920), “Time comparison of existing images and current images 925” (after step 920), “Calculate 3D model of cold storage room 930”, “Determine positions and free spaces 935”, “Output positions and free spaces 940”, and “Report detected changes 945”.

[0072] The approach presented here can enable the acquisition of additional information, such as depth information, positions, sizes, and / or size changes of food items within the refrigerator, in addition to image data captured by integrated cameras and controllable lighting. This allows for improved identification of food items within the refrigerator, as well as their position and characteristics—particularly their shape and / or surface texture—based on 2D image data.

[0073] Without significant effort or additional hardware costs, the described approach allows for the derivation of supplementary information about food stored in refrigerators with an integrated camera. Such supplementary information could include, but is not limited to: position detection, size and / or volume detection, change detection and / or movement detection – which can particularly relate to food shelf life – any available space for food and / or the fill level of the refrigerator, mold growth and / or fermenting dough, and additionally or alternatively, potential general improvements for subsequent object identification, for example, based on image data.

[0074] The approach presented here can be described as an alternative possibility for "level, position and change detection using shadow images in refrigerators" and can be implemented, for example, in a refrigerator with camera support or in a wine device with camera support. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] US 2020003484A1

[0003]

Claims

[1] Method (900) for detecting a position and / or a condition of an object (200) in an interior space (105) of a device (100), the method (900) comprising the following steps: - reading (910) at least one camera image (220) representing the object (200) in the interior space (105); and - analyzing (920) the camera image (220) in order to detect the position and / or the nature of the object (200) in the interior (105) of the device (100). [2] Method (900) according to claim 1, wherein in an illuminating step the object (200) is illuminated using a predefined illumination unit (205) and in the analyzing step (920) a shadow (600) of the object (200) is analyzed in order to detect the position and / or nature of the object (200). [3] Method (900) according to claim 2, wherein the step of illuminating is carried out using the illumination unit (205) with at least two light sources (215) installed at different locations in the interior (105) of the device (100), and wherein the step (920) of analyzing comprises an evaluation of a brightness profile, in particular of a core shadow and / or penumbra of the object (200). [4] Method (900) according to claim 3, wherein the step of illuminating is carried out using at least two light sources (215), wherein the two light sources (215) are arranged on a same inner side of the interior (105) of the device (100) or on different, in particular opposite, inner sides of the interior (105) of the device (100). [5] Method (900) according to claim 3 or 4, wherein in the illuminating step the at least two light sources (215) are activated at different times. [6] Method (900) according to one of claims 2 to 5, wherein in the step (920) of analyzing an object-free area in the interior (105) of the device (100) is detected. [7] Method (900) according to one of claims 1 to 6, wherein in the step (920) of analyzing, a size, a surface texture and / or a volume of the object (200) is recognized as the texture of the object (200). [8] Method (900) according to one of the preceding claims, wherein in the step (910) of reading in at least one further camera image (320) is read in, wherein the further camera image (320) was recorded by a further camera (115) which is arranged at a different position in the interior (105) of the device (100) than the camera (110) which provides the camera image (220) and / or wherein the further camera image (320) was recorded at a different time than the camera image (220), and wherein in the step (920) of analyzing the position and / or the nature of the object (200) is further recognized using the further camera image (320). [9] Method (900) according to claim 8, wherein in the step (920) of analyzing, the position and / or the nature of the object (200) is recognized from a three-dimensional image (500) of the object (200) in the interior (105) of the device (100) determined from the camera image (220) and the further camera image (320). [10] Method (900) according to one of claims 8 or 9, wherein in the step (910) of reading in a time period is read in which lies between the recording of the camera image (220) and the further camera image (320), wherein in the step (920) of analyzing the nature of the object (200) is recognized using the time period, in particular if the time period is greater than a predefined time period value. [11] Method (900) according to one of claims 8 to 10, wherein in the step (920) of analyzing, a condition of the object (200) is recognized if a position of the object (200) recognized from the camera image (220) corresponds within a tolerance range to a further position of the object (200) recognized from the further camera image (320). [12] Method (900) according to one of claims 8 to 11, wherein in the step (920) of analyzing, a condition of the object (200) is recognized if a shape, color and / or surface condition of at least one part of the object (200) recognized from the camera image (220) deviates by more than a tolerance range from a further shape, color and / or surface condition of at least one further part of the object (200) recognized from the further camera image (320). [13] Device (325) which is designed to carry out and / or control the steps (910, 920) of the method (900) according to one of the preceding claims in corresponding units. [14] Computer program product with program code for carrying out the method (900) according to one of claims 1 to 12, when the computer program product is executed on a device (325) according to claim 13. [15] Domestic appliance and / or commercial appliance (100), in particular a refrigerator, with a device (325) according to claim 13.

Images