Dishwasher and method for identifying objects in a dishwasher

Abstract

(Washing machine and method for identifying objects in a washing machine) A washing machine (105) comprises a wash chamber (115), a dish basket (120) for accommodating objects (125) which can be pulled out of the wash chamber (115), a camera (150) for acquiring images of the objects (125), and a processing device (145). Here, the processing device (145) is configured to determine three-dimensional information about the objects (125) for a first image and a second image of the camera (150), wherein the images are acquired at different positions of the dish basket (120) relative to the camera (150).

Description

Technical Field

[0001] This invention relates to dishwashers. In particular, this invention relates to the identification of information about objects placed in a dishwasher. Background Technology

[0002] The dishwasher includes a rinse chamber and a dish basket. Objects can be placed in the dish basket, and then cleaned in the rinse chamber.

[0003] US 8,696,827 B2 proposes a dishwasher with a camera to determine the arrangement of objects in the dish rack and to coordinate the cleaning process with the arrangement.

[0004] DE 10 2020 211 543 A1 relates to a dishwasher with a horizontally pull-out dish basket. A camera is configured to: acquire images of the dish basket at different horizontal positions. The images are stored in an associated storage location and are capable of subsequent processing.

[0005] US 2022 092 808 A1 proposes optically scanning an object in a dishwasher under different lighting conditions and creating a 3D image of the object based on the scanned images. Summary of the Invention

[0006] The objective of this invention is to provide an improved technique for providing information about objects arranged in a dishwasher for cleaning. This objective is achieved by means of the body of the independent claims. The dependent claims describe preferred embodiments.

[0007] According to a first aspect of the invention, a dishwasher includes: a rinsing chamber; a dish basket for containing an object, the dish basket being pullable out of the rinsing chamber; a camera for acquiring images of the object; and a processing device. Here, the processing device is configured to determine three-dimensional information about the object based on a first image and a second image from the camera; wherein images are acquired at different positions of the dish basket relative to the camera.

[0008] It has been recognized that a camera is sufficient to provide images from different perspectives, from which three-dimensional information can be determined. Preferably, the camera is attached in a fixed position relative to the rinsing chamber, enabling images to be provided along different straight lines as the dish rack is pulled horizontally out of the rinsing chamber. Conventional dishwashers are equipped with limited mechanisms that accommodate the camera.

[0009] The rinsing chamber can have a reflective inner surface. A camera can be configured to capture reflections of cutlery baskets and / or objects on the surface. Preferably, the camera is arranged or aligned such that both direct images and reflections can be identified in the image. Here, the direct images and reflections can contain different content.

[0010] According to another aspect of the invention, a system includes a dishwasher and a mobile device as described herein. The dishwasher is configured to transmit determined three-dimensional information to the mobile device, and the mobile device is configured to provide prompts for that information.

[0011] Mobile devices can include, in particular, a user's personal device, such as a smartphone, tablet, or laptop computer. Mobile devices can include a graphics output device on which a representation of the scanned object can be displayed. Optionally, additional information can also be provided to the user, particularly suggestions regarding potentially more advantageous locations for the scanned object.

[0012] According to another aspect of the present invention, a method for providing three-dimensional information about an object in a dish rack of a dishwasher using a camera includes the following steps: acquiring a first image of the object when the dish rack is in a first position relative to the camera; acquiring a second image of the object when the dish rack is in a second position relative to the camera; wherein the second position is different from the first position; determining three-dimensional information about the image; and providing the determined information.

[0013] This method can be executed by means of a dishwasher or system described herein. The processing device of the dishwasher can be configured to partially or completely execute the method. For this purpose, the processing device can be electronically configured and, for example, include a programmable microcomputer or microcontroller. The method can exist in the form of a computer program product having program code as a medium. The computer program product can be stored on a computer-readable data carrier. Features or advantages of the method can be transferred to the dishwasher or system, and vice versa.

[0014] Preferably, multiple images of the object are determined from different positions of the cutlery basket relative to the camera, and three-dimensional information is determined from the multiple images. The different positions of the cutlery basket relative to the rinsing chamber can be predetermined by the predetermined relative mobility of the cutlery basket. In an embodiment, the cutlery basket is arranged to be horizontally displaceable relative to the rinsing chamber by means of drawer slides or a similar construction. In the predetermined final position, the cutlery basket is housed in the rinsing chamber, allowing the rinsing chamber to be closed, typically by means of a flap that pivots about a horizontal axis.

[0015] In another embodiment, multiple cameras are provided, each constructing an image for scanning the object. Here, the cameras are arranged staggered relative to each other, so that they have different perspectives on the object. Images scanned by two or more cameras at different horizontal positions in the dish basket can effectively help create three-dimensional information of the scanned object.

[0016] Three-dimensional information can be determined classically using triangulation. Based on the known positions of an object in different images, corresponding optical features on the images can be determined and placed in a three-dimensional context. In the case of only two images, this is also called solid triangulation.

[0017] Preferably, three-dimensional information is determined by means of a neural radiance field (NeRF).

[0018] Neural Radiation Field (NeRF) is a deep learning-based approach for reconstructing a three-dimensional (3D) representation of a scene from two-dimensional (2D) images. Here, it refers to graphical primitives that can be optimized from a set of 2D images to generate a 3D scene. The NeRF model can learn the geometry of the scene, camera position, and the reflective properties of objects in the scene, thereby optionally also representing new views of the scene from new perspectives. Available software for implementing NeRF includes ProteusNeRF, hypernerf, or Nerfstudio.

[0019] To train the creation of 3D information about objects, reinforcement learning can be used on a kinematic neural network (KNN). The training data preferably includes images of objects typically cleaned in a dishwasher, such as dishes, cutlery, glasses, pots, pans, or cooking utensils. Each object can be optically scanned from different viewpoints to provide a set of images from which 3D information can be created. Here, the viewpoint of the set is preferably aligned with the object in a straight line. Objects can be scanned at different positions or from viewpoints along different lines to generate training data.

[0020] The position of the cutlery basket can be determined, for example, by means of a dedicated sensor. The sensor can be specifically configured to determine the horizontal position of the movable cutlery basket relative to the flushing chamber. However, such a sensor can be omitted by determining the position of the cutlery basket based on scanned images.

[0021] In a preferred embodiment, predetermined optical features of the dish basket are identified, wherein the position of the dish basket relative to the camera is determined with respect to the identified features. For determining the position, an image alone is sufficient. The dish basket can typically be manufactured based on a wire mesh and includes numerous edges or surfaces that can serve as optical features. For example, optical features can include the intersection of two wires, an angle, or a characteristic bend in the wire. The dish basket is fixedly associated with the dishwasher such that its optical properties are generally known and invariant. Alternatively, optical markings can also be attached or formed at the dish basket. For example, predetermined reflective markings can be subsequently attached to the dish basket. Alternatively, the dish basket can be specially shaped such that it forms optical markings that are easily identifiable in the image. In an embodiment, the optical markings of the dish basket can include the dishwasher manufacturer's trademark. Thus, additional protection against unintended use of the dish basket at the dishwasher can be achieved.

[0022] Preferably, at least one image is acquired while the dish basket is being pulled out or pushed into the rinsing chamber. This process can be performed frequently during normal operation of the dishwasher. Here, multiple images can be scanned, which can help determine three-dimensional information. The dish basket can be pulled out and pushed into the rinsing chamber multiple times after an object has been placed inside. Each time, an image of the object can be scanned, providing sufficient data for determining three-dimensional information. This method can determine the three-dimensional structure of an object from its relative motion with respect to the camera, and therefore can be called a "motion-recovery structure" method.

[0023] In another implementation, objects are classified. For classification, a dedicated or the same KNN used to create the 3D information can be used. Exemplary categories for objects include cutlery, pots and pans, glasses, plates, cups, frying pans, baking utensils, or cooking tools. The classification can be associated with the 3D information. Optionally, the determination of the 3D information can be further refined based on the determined object categories.

[0024] Optionally, the arrangement of objects in the rinsing chamber can be determined to improve the cleaning of objects in the dishwasher. The arrangement can include the state, position, or orientation of the objects. For example, it can be suggested that bowls placed open-to-the-top in the dish rack be turned upside down. Based on information associated with the categorized objects, it can also be determined where in the rinsing chamber the objects can be best handled. For example, objects comprising plastic parts can be advantageously arranged in corners during a high-temperature rinsing process to avoid deformation of the plastic. Utensils made of wood, porcelain, glass, steel, or cast iron can be placed in advantageous positions in the rinsing chamber. The dishwasher can include multiple dish racks, and the arrangement can extend over multiple dish racks. For example, it can be suggested that pots arranged in the upper dish rack are more advantageously arranged in the lower dish rack, where a more powerful cleaning can be performed.

[0025] Preferably, the three-dimensional information of multiple objects is determined, thereby enabling the determination of an optimal arrangement of the objects in the rinsing chamber. This improves the utilization of available space in the rinsing chamber, reduces or avoids undesirable mutual overlap of objects, improves cleaning results, and shortens cleaning time. Attached Figure Description

[0026] The invention will now be described in more detail with reference to the accompanying drawings, in which: Figure 1 A system with a dishwasher is shown; and Figure 2 A method for determining the three-dimensional information of an object in a dishwasher is shown. Detailed Implementation

[0027] Figure 1 The system 100 is shown, which includes a dishwasher 105 and a mobile device 110.

[0028] The dishwasher 105 includes a rinsing chamber 115 in which at least one dish basket 120 is arranged. As an example only, the dishwasher 105 shown has an upper dish basket, a middle dish basket, and a lower dish basket 120. The dish baskets 120 are each horizontally displaceable and attached to the dishwasher 105, allowing them to be pulled out of or pushed into the rinsing chamber 115.

[0029] As the dish baskets are pulled out of the rinsing chamber 115, the objects 125 to be cleaned can be inserted into one of the dish baskets 120. After all the dish baskets 120 are pushed into the rinsing chamber 115, the rinsing chamber can be closed by means of a flap 130 that flips upward about an axis arranged in the lower region of the dishwasher 105. After closing the rinsing chamber 115, water and detergent can be dispensed into the rinsing chamber 115. Water can be directed from below through the dish baskets 120 to the contained objects 125 by means of a spray arm 135.

[0030] The dishwasher 105 preferably includes a device 140 configured to determine and provide three-dimensional information about an object 125 contained within the dishwasher 105. For this purpose, the device 140 preferably includes a processing unit 145 and a camera 150. Also preferably, a communication device 155 is provided for communicating with a mobile device 110. The communication device 155 is preferably wireless and, for example, capable of establishing a Bluetooth (BT, BLE) or WLAN connection. If the mobile device 110 cannot be used, an output device can be provided instead of the communication device 155, the output device being configured to provide information to a human user of the system 100 or the dishwasher 105.

[0031] The device 140 is configured to: scan images of the object 125 in the dish rack 120 from different perspectives, and determine three-dimensional information about the object 125 based on the images. For this purpose, a three-dimensional model of the object 125 can be created. Furthermore, the object 125 can be identified or classified. Predetermined information can be associated with the identified object 125 or the category to which the object 125 is classified. For example, size, material, preferred treatment, incompatibility with a particular detergent, or other information that particularly affects the cleaning process in the dishwasher 105 can be associated with this information.

[0032] It is also proposed that, based on the identified object 125, its position and / or orientation within the rinsing chamber 115, it be determined whether other positions or orientations might be more advantageous for the cleaning process. For multiple objects 125, the arrangement of the objects 125 can be determined, allowing for an optimized cleaning process. For example, with the aid of a mobile device 110, prompts can be given to the user so that, after determination, the object 125 can be more advantageously arranged in the rinsing chamber 115 or the dish rack 120. For improved arrangement, in some cases it may be necessary to remove the object 125 from the rinsing chamber 115.

[0033] Figure 2 A flowchart illustrates an exemplary method 200 for determining three-dimensional information of an object 125 in a dishwasher 105. Method 200 can preferably be implemented using... Figure 1 The system 100 or dishwasher 105 will be used to perform this.

[0034] In step 205, a first image of the object 125 located in the dish basket 120 of the dishwasher 105 can be scanned using camera 150. In step 210, a first position of the dish basket 120 can be determined. To determine the position, optically recognizable features of the dish basket 120 can be identified on the first image. The orientation of the features on the image can infer the horizontal position of the dish basket 120 relative to the rinsing chamber 115. Optical features can include, for example, dedicated reflective markings or the characteristic shape of the dish basket 120. The dish basket 120 can be designed as a basket body, especially a wire basket body. The wire or corresponding mesh can here be formed, individually or in combination with other wires or meshes, into geometric features that can be well identified on the first image.

[0035] In step 215, a second image of the object 125 in the cutlery basket 120 can be scanned. In step 220, a second position of the cutlery basket 120 can be determined. Here, this can be performed as in step 210 regarding the first position.

[0036] In step 225, it can be determined whether the differences between the determined positions are large enough. If the positions are not sufficiently different from each other, for example, because they are completely indistinguishable, the second image can be discarded, and method 200 can continue to step 215.

[0037] In steps 205 to 225, more than two images at different locations within the dish basket 120 can also be acquired. During multiple iterations of steps 215 to 225, the dish basket 120 can be pulled out of or pushed into the rinsing chamber 115. For example, image acquisition can end when no further changes in the determined position occur within a predetermined time. Image determination can continue as long as the lid 130 of the dishwasher 105 is open.

[0038] If a sufficient number of images have been scanned, a three-dimensional model of object 125 can be determined in step 230 based on the acquired images. The three-dimensional model can be generated as a numerical model or, for example, a graphical representation. Preferably, the model is determined using a neural radiation field. For this purpose, a KNN (artificial neural network) can be provided, which is trained to determine the three-dimensional information of object 125 based on two-dimensional images from different perspectives of object 125.

[0039] In step 235, object 125 can be identified or classified. Information can be associated with the identified object 125 or its relevant category, and this information can also be used to create a 3D model.

[0040] In step 240, when the cutlery basket 120 is pushed into the rinsing chamber 115, the arrangement of the objects 125 in the rinsing chamber 115 can be determined. In step 245, the arrangement of multiple objects 125 in the rinsing chamber 115 can also be determined. Here, objects 125 arranged in different cutlery baskets 120 can be considered. The relative arrangement of different objects 125 to each other can be determined or considered.

[0041] In step 250, an optimal arrangement of one or more objects 125 can be determined. The arrangement can preferably be optimized with respect to the cleaning process. The objects 125 can be optimally arranged such that they are surrounded and rinsed by water in the rinsing chamber 115 in a correlated manner during the cleaning process. Mutual obstruction of the objects 125 can be avoided. Furthermore, the objects 125 can be arranged such that liquid can advantageously flow out of the objects 125 at the end of the rinsing process.

[0042] In step 255, a prompt can be determined, specifying how the objects 125 must be rearranged to achieve an improved arrangement. The prompt can be directed to the user of the dishwasher 105. Preferably, the prompt is output to the user via a mobile device 110. In embodiments, the prompt can be graphical. For example, animation can be provided showing how the objects 125 should be moved relative to the dish rack 120 to achieve the improved arrangement.

[0043] Figure Labels

[0044] 100 System

[0045] 105 Dishwasher

[0046] 110 Mobile Devices

[0047] 115 Rinse Room

[0048] 120 Cutlery Basket

[0049] 125 objects

[0050] 130 Flip

[0051] 135 spray arm

[0052] 140 devices

[0053] 145 Processing Unit

[0054] 150 cameras

[0055] 155 Communication devices

[0056] 200 methods

[0057] 205 Scan the first image

[0058] 210 Determine the first position of the cutlery basket.

[0059] 215 Scan the second image

[0060] 220 Determine the second position of the cutlery basket.

[0061] Is the positional difference 225 large enough?

[0062] 230 Determine the 3D model of the object

[0063] 235. Classifying objects

[0064] 240 Determine the arrangement of objects in a closed dishwasher.

[0065] 245. Determine the arrangement of multiple objects.

[0066] 250 Determine the optimal layout

[0067] 255. Provide prompts to users.

Claims

1. A dishwasher (105), comprising: - Rinse chamber (115); - A cutlery basket (120) for holding objects (125), the cutlery basket being able to be pulled out from the rinsing chamber (115); - A camera (150) for acquiring an image of the object (125); and - Processing device (145), the processing device being configured to determine three-dimensional information about the object (125) based on a first image and a second image from the camera (150); -The images are acquired at different positions of the cutlery basket (120) relative to the camera (150).

2. The dishwasher (105) according to claim 1, wherein, The rinsing chamber (115) has a reflective inner surface; wherein the camera (150) is configured to acquire the reflection of the rinsing basket and / or the object (125) at the surface.

3. A system (100) comprising a dishwasher (105) and a mobile device (110) according to claim 1 or 2, wherein, The dishwasher (105) is configured to transmit determined three-dimensional information to the mobile device (110), wherein the mobile device (110) is configured to provide an indication of the information.

4. A method (200) for providing three-dimensional information about an object (125) in a dish rack (120) of a dishwasher (105) using a camera (150), wherein, The method (200) includes the following steps: -When the cutlery basket (120) is in a first position relative to the camera (150), acquire (205) a first image of the object (125); -When the cutlery basket (120) is in a second position relative to the camera (150), acquire (215) a second image of the object (125); - Wherein, the second position is different from the first position; - Determine the three-dimensional information (230) for the image; and - Provide (255) confirmed information.

5. The method (200) according to claim 4, wherein, Multiple images of the object (125) are determined (205-225) from different positions of the cutlery basket (120) relative to the camera (150), and wherein the three-dimensional information for the multiple images is determined.

6. The method (200) according to any one of claims 4 or 5, wherein, The three-dimensional information (230) is determined by means of a neural radiation field.

7. The method (200) according to any one of claims 4 to 6, wherein, Identify predetermined optical features of the cutlery basket (120), and wherein, for the identified features, determine (210, 220) the position of the cutlery basket (120) relative to the camera (150).

8. The method (200) according to any one of claims 4 to 7, wherein, At least one image is acquired when the cutlery basket (120) is pulled out of or pushed into the rinsing chamber (115).

9. The method (200) according to any one of claims 4 to 8, wherein, The object (125) is classified (235).

10. The method (200) according to any one of claims 4 to 9, wherein, The arrangement of the object (125) in the rinsing chamber (115) is determined (250), the arrangement of which improves the cleaning of the object (125) in the dishwasher (105).

11. The method (200) according to any one of the preceding claims, wherein, The three-dimensional information of multiple objects (125) is determined, wherein an optimal arrangement of the objects (125) in the flushing chamber (115) is determined.

Images