Sorting equipment and sorting procedure for cutlery, as well as system

Abstract

A cutlery sorting device (110) for cutlery (120) comprises an optical scanning device (140) for scanning a cutlery basket (115); a movable gripper (125) for cutlery (120); and a processing unit (145). The processing unit (145) is configured to detect the cutlery (120) in the cutlery basket (115); select a detected piece of cutlery (120); and control the gripper (125) to grasp the selected piece of cutlery (120) and transfer it from the cutlery basket (115) to a predetermined storage location (135).

Description

[0001] The present invention relates to the sorting of cutlery. In particular, the invention relates to a sorting device for sorting cutlery from a cutlery basket, as well as a corresponding sorting method.

[0002] A dishwasher is designed to hold and clean kitchen utensils. Dishes and pots can be arranged in a dish rack, while cutlery can be stored in a cutlery drawer or basket. The cutlery drawer allows cutlery to be sorted while loading the dishwasher, so it can be removed and stored neatly after the cleaning cycle. However, sorting can be tedious or unpleasant, especially if the cutlery is heavily soiled. Furthermore, the cutlery drawer can take up valuable vertical space in the dishwasher, which is then no longer available for items such as long-stemmed glasses or tall pots.

[0003] The cutlery basket is a space-saving alternative that allows for quicker and, in particular, less organized placement of cutlery. However, the cutlery must be sorted after the cleaning cycle in order to be stored neatly, for example, in a cutlery drawer.

[0004] WO 2021 006 306 A1 proposes a robot designed to pick dishes from a conveyor belt that is part of a dishwashing line. A cutlery sorting system is previously known from US 2021 / 0001488 A1.

[0005] DE 10 2020 204 083 A1 concerns the dynamic downloading of program code tools to handle dishes at a dishwasher using a robot.

[0006] Familiar techniques for assisting with loading or unloading a dishwasher typically focus on dishes and neglect cutlery. Automatic handling of cutlery in a cutlery basket is hampered by the fact that cutlery is often found in unpredictable positions or configurations, making it difficult to grasp and remove from the basket. Frequently, cutlery is caught, tangled, or wedged in the basket after the cleaning cycle, making automatic removal nearly impossible.

[0007] One of the problems underlying the present invention is to provide an improved technique for sorting cutlery from a cutlery basket. The invention solves this problem by means of the subject matter of the independent claims. Dependent claims describe preferred embodiments.

[0008] According to a first aspect of the present invention, a cutlery sorting device comprises the features of independent claim 1, in particular an optical scanning device for scanning a cutlery basket; a movable gripper for cutlery; and a processing unit. The processing unit is configured to detect cutlery in the cutlery basket; to select a detected piece of cutlery; and to control the gripper to grasp the selected cutlery and move it from the cutlery basket into a predetermined storage location. Furthermore, the processing unit is configured to select a piece of cutlery based on the degree to which the cutlery is intertwined with other cutlery and / or with the cutlery basket.

[0009] The cutlery can be classified, and cutlery of different classes can be collected in assigned trays or compartments. This allows cutlery of different classes to be sorted into their respective designated trays or compartments.

[0010] The cutlery basket is preferably designed for cleaning cutlery in a dishwasher. The dishwasher is preferably a household appliance suitable for use in a private household. The cutlery basket can be an integral part of the dishwasher. In one embodiment, the sorting device is designed to remove the cutlery from the cutlery basket while the basket is in the dishwasher. In another embodiment, the cutlery basket can be removed from the dishwasher and placed in a working area of ​​the sorting device. The sorting device can be used to perform the task of removing and sorting cutlery after a dishwasher cleaning cycle, a task many users find tedious. The storage area can, for example, include a cutlery drawer in which the cutlery can be stored until the next use. Other storage options are also possible.

[0011] The cutlery basket is preferably designed to hold cutlery in a randomly arranged manner. The cutlery is preferably positioned in the basket in essentially vertical positions. The cutlery is typically elongated, and the cutlery basket is preferably shaped and dimensioned such that an elongated section of the cutlery protrudes from the top of the basket. The cutlery can comprise a working section and a handle section, with the handle section preferably oriented downwards in the basket to facilitate cleaning of the upward-protruding working section. In another embodiment, however, the cutlery can also be oriented upwards.

[0012] The cutlery preferably comprises eating utensils, which may be part of a place setting. For example, the cutlery may include a knife, fork, spoon, or chopsticks. It is further preferred that the cutlery does not include cooking utensils. Cooking utensils are defined as handheld devices used during food preparation, whereas cutlery is generally used only during eating.

[0013] The sorting device can be operated independently of any other appliance in a household. According to another aspect of the present invention, a dishwasher comprises a sorting device as described herein. The sorting device can be integrated into the dishwasher so that, after a cleaning cycle is complete, the cleaned cutlery can be automatically unloaded and sorted. According to yet another aspect, the invention comprises a system consisting of a dishwasher and a sorting device.

[0014] According to a second aspect of the present invention, a method for sorting cutlery comprises the features of independent claim 6, in particular steps of scanning a cutlery basket; detecting cutlery in the cutlery basket; selecting a detected piece of cutlery; and controlling a gripper to grasp the selected cutlery and move it from the cutlery basket into a predetermined storage location. Cutlery is selected based on the degree to which it is intertwined with other cutlery and / or with the cutlery basket.

[0015] The method can preferably be carried out using a sorting device described herein. Part of the method can be in the form of a computer program product with program code means. A processing device can be implemented electronically and may, for example, include a programmable microcomputer or microcontroller. The computer program product can be stored on a computer-readable data carrier. Features or advantages of the method can be transferred to the sorting device, the system, or the dishwasher, and vice versa.

[0016] The scanning process is preferably carried out in such a way as to obtain depth information about the position and orientation of cutlery within the cutlery basket. For this purpose, a depth camera, a stereo camera, or a time-of-flight (ToF) camera can be used, for example. Cutlery recognition within the cutlery basket can be achieved using pattern recognition principles via an artificial neural network (ANN). The ANN is trained to recognize and classify predetermined cutlery.

[0017] A piece of cutlery can be identified based on a section protruding from the cutlery basket. A section of the cutlery located within the basket can then be used to determine or extrapolate the location of the identified piece. This allows for a more reliable determination of the cutlery's position or orientation.

[0018] When identifying a piece of cutlery, it's possible to determine the probability of the identification being correct. This probability can be expressed as a confidence level. The higher the confidence level, the higher the probability of the identification being correct.

[0019] In one embodiment, cutlery is selected based on the confidence level with which it was recognized. Cutlery recognized with high confidence can be preferentially taken first, while cutlery recognized with lower confidence can be taken later. This facilitates grasping and handling the selected cutlery.

[0020] It is possible to determine how strongly one piece of cutlery is entangled or wedged with another. This is achieved by detecting multiple pieces of cutlery in the cutlery basket. Entanglement typically occurs on a section of cutlery that lies within the basket and cannot be directly visualized. However, the shape of a detected piece of cutlery can be known, allowing the degree of entanglement to be determined. Preferably, a piece of cutlery with a low degree of entanglement is selected. A piece of cutlery with a higher degree of entanglement can be removed from the basket at a later time. The degree of entanglement of a piece of cutlery can decrease when other cutlery is removed, as entangled pieces are eliminated. This allows for smoother and more error-free cutlery removal.

[0021] Similarly, cutlery can be selected based on the degree to which it is interwoven with the cutlery basket. Again, it is preferable to start with cutlery that has the lowest possible degree of interweaving.

[0022] The inverse of the degree of interconnectedness can be called the degrees of freedom. For each piece of cutlery detected in the cutlery basket, the product of the degrees of freedom and the confidence with which the cutlery could be detected can be calculated. The cutlery with the highest product can be removed from the cutlery basket first.

[0023] A trajectory can be determined for a piece of cutlery to be removed from the cutlery basket, along which the cutlery can be removed as efficiently as possible. The shape and / or position of the cutlery can be taken into account. Furthermore, one or more surrounding objects, especially other cutlery or the cutlery basket itself, can be considered. The trajectory can be determined so that the cutlery is pulled out of the basket essentially along its longitudinal axis. Collisions with other objects can be avoided if possible. A Gaussian process can be used to evaluate a trajectory.

[0024] In yet another embodiment, a gripping point is defined at which the cutlery can be grasped with the gripper. The trajectory with respect to this gripping point can also be determined. One or more gripping points can be provided on a piece of cutlery, at which the gripper can advantageously be positioned. The trajectory can be determined such that the gripper, or any part connected to it, avoids colliding with or scraping along any other object.

[0025] In another embodiment, the gripper can oscillate perpendicular to the selected trajectory while the chosen cutlery is being pulled out. The frequency or amplitude of the oscillation can be selected so that tangled cutlery can be gently removed from other cutlery or the cutlery basket. Even complex tangles of cutlery can be untangled safely and without damaging any of the pieces. Colloquially, this oscillation might be described as a jerking motion.

[0026] Generally, if the removal of a selected piece of cutlery is unsuccessful and the cutlery cannot be removed from the cutlery basket with the gripper, another attempt is made after at least one other piece of cutlery has been removed from the basket. This process can be repeated until all cutlery has been individually removed from the basket and placed in the tray. In the tray, the cutlery can be sorted into categories to which each piece is assigned. Examples of such categories include teaspoons, soup spoons, cake forks, forks, knives, and chopsticks. Other categories may also be predefined.

[0027] To detect an unsuccessful attempt, a force sensor can be used, for example, to detect mechanical resistance encountered during the removal process. Additionally, an unintentional movement of the cutlery basket can be detected via a camera.

[0028] A recognized piece of cutlery can be assigned a class. For elements within a class, the tendency to become entangled can be known. Cutlery from a class whose elements tend to become less entangled can be preferentially pulled out of the cutlery basket first.

[0029] For example, a table knife typically has few protrusions or changes in dimensions along its length, so it is unlikely to become entangled. For this reason, it can be advantageous to remove knives from the cutlery basket before taking out other cutlery. Experience has shown that forks have the strongest tendency to become entangled with other cutlery or the cutlery basket. In one embodiment, forks can be removed from the cutlery basket only after as much of the other cutlery as possible has been taken out.

[0030] Non-limiting embodiments of the invention are now described in more detail with reference to the accompanying figures, in which: Figure 1 shows a household with a dishwasher and a cutlery sorting device; Figure 2 shows a method for sorting cutlery from a cutlery basket; and Figure 3 shows exemplary cutlery. represent.

[0031] Figure 1 Figure 100 shows an exemplary household with a dishwasher 105 and a sorting device 110. The sorting device 110 can operate independently or be part of the dishwasher 105.

[0032] The dishwasher 105 is designed to clean kitchen utensils. A basket or pull-out tray for holding kitchen utensils is included. Figure 1Not shown. A cutlery basket 115 is preferably provided for holding cutlery, which can optionally be removed from the dishwasher 105. The cutlery basket 115 is usually arranged in a dish trolley that can be pulled horizontally out of a processing area of ​​the dishwasher 105.

[0033] The cutlery basket 115 is designed to hold cutlery 120. The cutlery 120 preferably comprises eating utensils or tableware that can be used by one person for eating. Cutlery 120 is typically used as part of a place setting in a household 100. Examples of cutlery 120 include chopsticks, knives, forks, or spoons. Cutlery 120 can be divided into different classes. Typically, a class comprises cutlery 120 that is of the same type, size, intended for the same purpose, or made of the same material. The classes can be differentiated to varying degrees. In one embodiment, knives of all types can be grouped together in one class; in another embodiment, for example, table knives, appetizer knives, fish knives, fruit knives, and butter knives can be distinguished.

[0034] The sorting device 110 comprises a gripper 125, which is exemplified as being attached to an arm 130. The arm 130, or a corresponding device, is configured to move the gripper 125 relative to the cutlery basket 115. The gripper 125 is typically designed in the form of pliers or tweezers. In another embodiment, for example, a pneumatic gripper, a magnetic gripper, or a thermoelastic gripper may also be provided. The gripper 125 is configured to grasp a piece of cutlery 120 so that the cutlery 120 can be pulled out of the cutlery basket 115. For this purpose, the gripper 125 can be moved appropriately by means of the arm 130. The cutlery 120 can then be moved to a tray 135, where it can be placed. The tray 135 can facilitate the separation of cutlery 120 of different classes, for example, by providing different compartments, as shown in Figure 1As indicated, in another embodiment the sorting device 100 is integrated with the storage tray 135, for example in the manner of smart furniture. The sorting device can be part of a mobile autonomous robot.

[0035] The sorting device 110 further comprises an optical scanning device 140, which is preferably configured to perform a three-dimensional scan of cutlery 120 in the cutlery basket 115. For this purpose, the scanning device 140 can, for example, comprise a stereo camera or a time-of-flight (ToF) camera. In another embodiment, the scanning device 140 comprises a LiDAR sensor, a radar sensor, or an ultrasonic sensor. Combinations of different sensors are also possible. Thus, the scanning device 140 can provide an optical representation, i.e., an image, of cutlery 120 in the cutlery basket 115, with each individual pixel of cutlery 120 being assigned a spatial position.

[0036] The scanning device 140 can be attached to the arm 130 or the gripper 125. The position of a pixel of a cutlery 120 relative to the scanning device 140 can be combined with the position of the arm 130 to determine the position relative to a surrounding reference system in which the dishwasher 105 and / or the shelf 135 may be located.

[0037] A processing unit 145 is designed to evaluate a scan from the scanning device 140 and to control the gripper 125 or the arm 130 such that a piece of cutlery 120 is removed from the cutlery basket 115 and placed in the tray 135. For this purpose, the processing unit 145 can include an artificial neural network 150, which is trained to recognize a piece of cutlery 120 based on an optical scan. Classification can be performed simultaneously with the recognition. The processing unit 145 can generate a geometric model of the recognized cutlery 120 in the cutlery basket 115. The relationship of a piece of cutlery 120 with another piece of cutlery 120 or with the cutlery basket 115 can be determined based on this model. In one embodiment, a geometric shape is stored for each known piece of cutlery 120. The model can be determined based on the geometric shapes of the recognized cutlery 120.Regarding a geometric shape, a center of gravity or mass may also be specified. Furthermore, one or more gripping points may be indicated at which the gripper 125 can advantageously grasp and hold the cutlery 120.

[0038] Data relating to the experiences of a sorting device can be sent to a central data lake maintained by the sorting device manufacturer. By analyzing data from many sorting devices, the algorithms, strategies, and / or behaviors of the sorting device can be continuously improved.

[0039] Figure 2 Figure 1 shows a flowchart of an exemplary process 200 for sorting cutlery 120 from a cutlery basket 115. The process 200 can be carried out in particular by means of a sorting device 110.

[0040] In step 205, a cutlery basket 115 containing cutlery 120 can be scanned. The scanning is preferably optical, resulting in an image. In step 210, depth information for pixels in the image can be determined. For this purpose, two images taken from different perspectives, for example using a stereo camera, can be processed together. Alternatively, depth information can also be provided by a sensor other than an optical one.

[0041] In step 215, cutlery 120 can be detected in cutlery basket 115. The cutlery 120 is typically arranged in a disordered manner and essentially vertically in the cutlery basket 115, with only a portion protruding. Cutlery 120 can be detected based on an optical scan of this protruding portion. A confidence level can be determined during detection, reflecting the quality of the detection. The higher the confidence level, the greater the probability that the assigned detection correctly corresponds to the detected object.

[0042] In step 220, the degree of interlocking of a piece of cutlery 120 can be determined. The degree of interlocking indicates how strongly the piece of cutlery 120 interacts with other pieces of cutlery 120 or the cutlery basket 115. The mechanical force required to pull out the piece of cutlery 120 may depend on the degree of interlocking. In some cases, the degree of interlocking may be so high that pulling out a piece of cutlery 120 is impossible.

[0043] In step 225, a detected piece of cutlery 120 can be selected from the cutlery basket 115. One or more criteria can be used for this. It is preferred that a piece of cutlery 120 be selected whose degree of entanglement is as low as possible. At the same time, it is preferred that a piece of cutlery 120 be selected whose confidence level is as high as possible. Furthermore, it can be taken into account that cutlery 120 of a predetermined class is preferentially selected. For this purpose, a tendency or inclination to entangle or wedge itself with another piece of cutlery 120 or the cutlery basket 115 can be assigned to the class. A piece of cutlery 120 of a class with a low tendency of this kind can be preferentially selected. Furthermore, it can be taken into account how well a piece of cutlery 120 can be gripped and / or moved using the gripper 125. For this purpose, it can be assessed whether a predetermined gripping point on a piece of cutlery 120 is accessible.The more accessible a gripping point is, the higher the associated gripping security, or the lower the collision of the gripper 125 with another cutlery 120, the more preferable a cutlery 120 can be selected.

[0044] In step 230, the selected cutlery 120 can be pulled out of the cutlery basket 115. For this purpose, the cutlery 120 can be pulled along a predetermined trajectory, which can be assigned to the cutlery 120 or determined dynamically.

[0045] In step 235, the removed cutlery 120 can be moved to the tray 135 and placed there. Cutlery 120 of different predetermined classes can be appropriately grouped in the tray 135.

[0046] Steps 205 to 235 can be executed in a loop to transfer one piece of cutlery 120 after another from the cutlery basket 115 to the tray 135. Optionally, an item other than cutlery 120, identified as being in the cutlery basket 115, can be removed and placed in a predetermined location. This allows, for example, the removal and collection of uncategorizable kitchen utensils. It should be noted that not all steps 205 to 235 need to be performed in every cycle. For example, several items can be removed from the cutlery basket 115 in succession without having to repeat steps 205 to 220.

[0047] If there are no more pieces of cutlery 120 in the cutlery basket 115, the process can proceed from step 215 to step 240, in which it is determined that no further pieces of cutlery 120 are detectable. In this case, the procedure 200 can end in step 245.

[0048] Figure 3 Figure 1 shows exemplary cutlery 120 in two exemplary embodiments. A fork 305 with a first trajectory 310 and a knife 315 with a second trajectory 320 are shown.

[0049] A trajectory 310, 320 can be fixedly assigned to the respective cutlery 305, 315 or dynamically determined. The trajectory 310, 320 preferentially follows a shape of the cutlery 305, 315. Since the fork 305 is curved along its longitudinal axis, the assigned first trajectory 310 can also be curved. The second trajectory 320 can, in particular, follow a shape of the fork 305 in the area of ​​its tines, where entanglement with another piece of cutlery 120 or the cutlery basket 115 is most likely. The knife 315, on the other hand, is essentially straight, and the second trajectory 320 is also straight. Hooking, wedging, or entanglement of the knife 315 is least likely with a straight direction of movement. Trajectories can be evaluated, for example, using a Gaussian process. Reference sign

[0050] 100 Household 105 Dishwasher 110 Sorting device 115 Cutlery basket 120 Cutlery 125 Gripper 130 Arm 135 Storage 140 Scanning device 145 Processing device 150 Artificial neural network (ANN) 200 Procedure 205 Scan cutlery basket 210 Determine depth information 215 Detect cutlery 220 Determine degree of entanglement 225 Select cutlery 230 Pull out selected cutlery 235 Place pulled-out cutlery 240 No further cutlery detected 245 End 305Fork 310first trajectory 315Knife 320second trajectory

Claims

1. Sorting device (110) for cutlery (120), wherein the sorting device (110) comprises the following: - an optical scanning apparatus (140) for scanning a cutlery basket (115); - a moveable gripper (125) for an item of cutlery (120); and - a processing facility (145), which is designed to identify cutlery (120) in the cutlery basket (115); to select an identified item of cutlery (120); and to control the gripper (125) in order to grip the selected item of cutlery (120) and to move the same out of the cutlery basket (115) into a predetermined tray (135); characterised in that - the processing facility (145) is designed to select an item of cutlery (120) on the basis of a degree with which the item of cutlery (120) is interwoven with another item of cutlery (120) and / or with the cutlery basket (115).

2. Sorting device (110) according to claim 1, wherein the cutlery basket (115) is designed to clean the item of cutlery (120) in a dishwasher (105).

3. Sorting device (110) according to claim 1 or 2, wherein the cutlery basket (115) is designed to accommodate cutlery (120) in an unordered manner and in essentially vertical positions.

4. Sorting device (110) according to one of the preceding claims, wherein the item of cutlery (120) comprises a knife (315), a fork (305), a spoon or a chopstick.

5. Dishwasher (105) comprising a sorting device (110) according to one of the preceding claims.

6. Method (200) for sorting cutlery (120); wherein the method (200) comprises the following steps: - scanning (205) a cutlery basket (115); - identifying (215) cutlery (120) in the cutlery basket (115); - selecting (225) an identified item of cutlery (120); and - controlling (230) a gripper (125), in order to grip the selected item of cutlery (120) and to move the same out of the cutlery basket (115) into a predetermined tray (135); characterised in that - an item of cutlery (120) is selected (225) on the basis of a degree with which the item of cutlery (120) is interwoven with another item of cutlery (120) and / or with the cutlery basket (115).

7. Method (200) according to claim 6, wherein an item of cutlery (120) is identified (215) on the basis of a segment which projects from the cutlery basket (115).

8. Method (200) according to one of claims 6 or 7, wherein an item of cutlery (120) is selected (225) on the basis of a confidence with which the item of cutlery (120) has been identified.

9. Method (200) according to one of claims 6 to 8, wherein with respect to a position of the item of cutlery (120) and surrounding objects, a trajectory (310, 320) is determined, along which the item of cutlery (120) can be moved as successfully as possible out of the cutlery basket (115).

10. Method (200) according to claim 9, wherein a handle is determined at which the cutlery (120) can be gripped with the gripper (125); wherein the trajectory (310, 320) is determined with respect to the handle.

11. Method (200) according to one of claims 9 or 10, wherein the gripper (125) is oscillated at a right angle to the determined trajectory (310, 320) when the selected item of cutlery (120) is being removed.

12. Method (200) according to one of claims 6 to 11, wherein a category is assigned to an identified item of cutlery (120), and wherein an item of cutlery (120) of one category, the elements of which tend to interweave less, is preferably removed first from the cutlery basket (115).

13. System which comprises a dishwasher (105) and a sorting apparatus (110) which is separate thereto according to one of claims 1 to 4.

Images