Waste sorting system reflecting picking difficulty

Abstract

A waste sorting system reflecting picking difficulty according to an embodiment of the present invention comprises: an image acquisition unit for collecting images of waste moving on a conveyor; a target object determination unit for determining pieces of the waste as being target objects to be sorted or non-target objects not to be sorted; and a picking difficulty determination unit that determines picking difficulty, which is the difficulty level of the task of gripping the target objects with the robot arm, according to the type of gripper the robot arm is provided with.

Description

Waste sorting system reflecting picking difficulty

[0001] The present invention relates to a waste sorting system that reflects picking difficulty, and more specifically, to a waste sorting system that reflects picking difficulty calculated according to the type of gripper of a robot arm.

[0002] This patent application claims priority to Korean Patent Application No. 10-2024-0140413 filed with the Korean Intellectual Property Office on October 15, 2024, and the disclosures of said patent application are incorporated herein by reference.

[0003] Large amounts of waste are generated daily from households, factories, restaurants, and other sources. While many people separate food waste from general waste, recyclable waste is still being disposed of mixed with general waste, requiring manual labor to separate it.

[0004] Recently, waste sorting devices capable of sorting waste without human intervention are being introduced. These devices utilize artificial intelligence to identify waste and employ robotic arms to sort it.

[0005] However, these waste sorting devices have a problem in that the success rate of waste sorting varies depending on the type of gripper equipped by the robot arm to grasp the waste.

[0006] To solve the aforementioned problems, the present invention aims to provide a waste sorting system that incorporates picking difficulty by using artificial intelligence to calculate the difficulty of the waste picking task based on the type of gripper equipped in the robot arm, and controlling the robot arm by reflecting the difficulty.

[0007] The waste sorting system reflecting picking difficulty according to the present invention may include: an image collection unit that collects waste images of waste moving on a conveyor; a target object determination unit that determines the waste as a target object to be classified and a non-target object not to be classified; a picking difficulty determination unit that determines the picking difficulty, which is the difficulty of the task of a robot arm grasping the target object, according to the type of gripper equipped by the robot arm; and a robot arm control unit that designates a robot arm to perform the task of grasping the target object according to the determined picking difficulty.

[0008] The above image collection unit may include a vision sensor capable of acquiring the waste image.

[0009] The above gripper may be either a suction type gripper that grips the target object using air suction force or a finger type gripper that grips the target object using a finger structure.

[0010] The robot arm may be equipped with at least one of the suction type gripper and the finger type gripper.

[0011] The above-mentioned picking difficulty determination unit may include an artificial intelligence model trained with data including waste images in which the picking difficulty is labeled according to a preset picking difficulty standard table.

[0012] The above picking difficulty standard table may include multiple classes, each having its own standard.

[0013] The above multi-class may include a first class based on the type of the gripper, a second class based on the shape of the waste, a third class based on the material composition of the waste, and a fourth class based on the contamination level of the waste.

[0014] The above picking difficulty determination unit may change the above picking difficulty standard table if the robot arm designated according to the determined picking difficulty fails to grasp the above target object and the failure rate exceeds a preset standard.

[0015] The robot arm control unit may designate a gripper among the grippers equipped by the robot arm to perform the task of gripping the target object.

[0016] According to the present invention as described above, the following effects are achieved.

[0017] The present invention can automatically calculate the picking difficulty of a task to grasp a target object to be classified among waste, and can designate an optimal gripper for classification by considering the picking difficulty.

[0018] The present invention can efficiently classify waste by performing a classification operation using an optimal gripper for gripping target objects.

[0019] The present invention can automatically change the picking difficulty standard table for calculating picking difficulty, thereby reducing the classification failure rate.

[0020] In addition, other features and advantages of the present invention may be newly identified through the embodiments of the present invention.

[0021] FIG. 1 is a drawing showing a waste classification device according to an embodiment of the present invention.

[0022] FIG. 2 is a diagram showing the configuration of a waste classification system reflecting picking difficulty according to an embodiment of the present invention.

[0023] FIG. 3 is a drawing showing the types of grippers according to an embodiment of the present invention.

[0024] FIG. 4 is a drawing showing a waste sorting device equipped with a plurality of robot arms according to an embodiment of the present invention.

[0025] FIG. 5 is a drawing showing a waste sorting device equipped with a plurality of grippers according to an embodiment of the present invention.

[0026] Figure 6 is a drawing showing a picking difficulty standard table according to an embodiment of the present invention.

[0027] It should be noted that in assigning reference numbers to the components of each drawing in this specification, identical components are given the same number as much as possible, even if they are shown in different drawings.

[0028] Meanwhile, the meaning of the terms described in this specification should be understood as follows.

[0029] A singular expression should be understood to include a plural expression unless the context clearly defines otherwise, and terms such as "first," "second," etc., are intended to distinguish one component from another, and the scope of rights shall not be limited by these terms.

[0030] Terms such as "include" or "have" should be understood as not excluding in advance the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0031] Hereinafter, preferred embodiments of the present invention designed to solve the above problems will be described in detail with reference to the attached drawings.

[0032] In the following, the waste classification system reflecting picking difficulty will be explained by abbreviating it as a waste classification system.

[0033] FIG. 1 is a drawing showing a waste classification device according to an embodiment of the present invention.

[0034] Referring to FIG. 1, the waste classification system (1000) can control the waste classification device by being connected to the waste classification device via wired or wireless connection.

[0035] A waste sorting device may include a robot arm (2000) and a conveyor (3000) as means for automatically sorting waste (10).

[0036] The conveyor (3000) can transport waste (10) to the robot arm (2000) of the waste sorting device. The waste sorting system (1000) can identify the target object to be sorted among the waste (10) and transmit a control signal to the robot arm (2000) to grasp the target object. The robot arm (2000) can grasp the target object according to the control signal and move it to a designated collection bin.

[0037] FIG. 2 is a diagram showing the configuration of a waste classification system reflecting picking difficulty according to an embodiment of the present invention.

[0038] Referring to FIG. 2, a waste classification system (1000) according to an embodiment of the present invention may include an image collection unit (1100), a target object identification unit (1200), a picking difficulty determination unit (1300), and a robot arm control unit (1400).

[0039] The image collection unit (1100) can collect images of waste moving on the conveyor (3000). As illustrated in FIG. 1, the image collection unit (1100) may include a vision sensor (1110) capable of acquiring images of waste.

[0040] The image collection unit (1100) can collect still images and videos of waste through a vision sensor (1110). The image collection unit (1100) can collect at least one of 2D and 3D images, and the type and category of the images are not limited.

[0041] The vision sensor (1110) can be any one of a camera, an RGB sensor, a TOF, or a LiDAR, and can be appropriately selected. The type and variety of the vision sensor (1110) are not limited.

[0042] According to an embodiment of the present invention, there may be one or more vision sensors (1110). For example, the image collection unit (1100) may include a first vision sensor (1110a) installed within the working area of ​​a robot arm and a second vision sensor (1110b) installed at the front of a conveyor (3000).

[0043] The target object determination unit (1200) can determine the waste as a target object to be classified and a non-target object not to be classified. The target object determination unit (1200) may include an artificial intelligence model for determining the target object.

[0044] For example, deep learning or neural networks may be used as artificial intelligence models for identifying target objects. Neural network models may include, but are not limited to, various types of models such as Convolutional Neural Networks (CNN), Region with Convolutional Neural Network (R-CNN), Region Proposal Network (RPN), Recurrent Neural Network (RNN), Stacking-based Deep Neural Network (S-DNN), State-Space Dynamic Neural Network (S-SDNN), Deconvolution Network, Deep Belief Network (DBN), Restructured Boltzmann Machine (RBM), Fully Convolutional Network, Long Short-Term Memory Network (LSTM), and Classification Network, such as GoogleNet, AlexNet, and VGG Network.

[0045] The picking difficulty determination unit (1300) can determine the picking difficulty, which is the difficulty of the task of the robot arm (2000) grasping a target object, according to the type of gripper equipped by the robot arm (2000).

[0046] As illustrated in FIG. 1, the robot arm (2000) may be equipped with a gripper (2100) on one side, which is a means for gripping a target object. The gripper (2100) may grip the target object using air suction force, magnetic force, etc., or by using various types of structures.

[0047] However, waste materials vary in countless types depending on their material and shape, as well as in levels of contamination and damage. Depending on these types and levels of contamination, the gripper (2100) may be able to easily grasp the target object, or it may be difficult to grasp it.

[0048] FIG. 3 is a drawing showing the types of grippers according to an embodiment of the present invention.

[0049] Referring to FIG. 3, the gripper (2100) according to an embodiment of the present invention may be either a suction type gripper that grips a target object using air suction force or a finger type gripper that grips a target object using a finger structure.

[0050] For example, as illustrated in FIG. 3(a), a suction type gripper (2100) that grips a target object using an air suction method can easily grip a flat target object. On the other hand, a gripper (2100) that grips a target object using an suction method may have difficulty gripping a target object with a curved surface, such as a water bottle.

[0051] As another example, as illustrated in FIG. 3(b), a gripper (2100) that grasps a target object using a finger structure can easily grasp a target object such as a water bottle. On the other hand, a flat and slab-like target object may be difficult to grasp with a gripper (2100) using a finger structure.

[0052] The picking difficulty according to an embodiment of the present invention may be a measure indicating the difficulty of the task of the gripper (2100) grasping a target object as described above. The picking difficulty may be determined as a specific numerical value or as a plurality of steps, but is not limited thereto.

[0053] For example, picking difficulty can be set to three levels: easy, normal, and difficult. As another example, picking difficulty can be set to a score from 0 to 10. The closer to 10 points, the more difficult the task of gripping can be, and the closer to 0 points, the easier the task of gripping can be.

[0054] The robot arm control unit (1400) can designate a robot arm to perform the task of grasping a target object according to the picking difficulty determined by the picking difficulty determination unit (1300).

[0055] FIG. 4 is a drawing showing a waste sorting device equipped with a plurality of robot arms according to an embodiment of the present invention.

[0056] Referring to FIG. 4, the waste sorting device may be equipped with a plurality of robot arms (2000). Each robot arm (2000) may be equipped with at least one of a suction type gripper and a finger type gripper.

[0057] For example, the waste sorting device may be equipped with a robot arm (2000a) equipped with a suction-type gripper (2100a) and a robot arm (2000b) equipped with a finger-type gripper (2100b). The picking difficulty determination unit (1400) may determine the picking difficulty of a target object and transmit it to the robot arm control unit (1400). The robot arm control unit (1400) may receive the picking difficulty and designate a robot arm suitable for gripping the target object.

[0058] Specifically, when the picking difficulty of the target object is Normal for the suction type gripper, the robot arm control unit (1400) can designate the robot arm (2000a) on which the suction type gripper (2100a) is installed as the robot arm to grasp the target object.

[0059] FIG. 5 is a drawing showing a waste sorting device equipped with a plurality of grippers according to an embodiment of the present invention.

[0060] Referring to FIG. 5, one robot arm (2000) may be equipped with different types of grippers (2100). A robot arm control unit (1400) according to an embodiment of the present invention may designate a gripper (2100) among the grippers (2100) equipped by the robot arm (2000) to perform the task of gripping a target object.

[0061] For example, the robot arm (2000) may be equipped with a suction type gripper (2100a) and a finger type gripper (2100b). The robot arm control unit (1400) may specify that the finger type gripper (2100b) grasps a target object according to the picking difficulty received from the picking difficulty determination unit (1300).

[0062] A robot arm (2000) according to an embodiment of the present invention may be provided with means for selecting a gripper (2100) to grasp a target object among different types of grippers (2100). For example, a suction type gripper (2100a) and a finger type gripper (2100b) may be attached to a rotatable disc. A robot arm control unit (1400) may select a gripper (2100) to grasp a target object by rotating the disc.

[0063] As another example, a suction type gripper (2100a) and a finger type gripper (2100b) can be attached to a structure in which the height of the gripper's Z-axis can be adjusted. A robot arm control unit (1400) can drive the structure to bring the gripper (2100) to grasp the target object closer to the target object.

[0064] According to an embodiment of the present invention, the picking difficulty determination unit (1300) may include an artificial intelligence model trained with data including waste images in which the picking difficulty is labeled according to a preset picking difficulty standard table.

[0065] Specifically, waste within a waste image can be labeled with a picking difficulty. Since the picking difficulty varies depending on the type of gripper, a single piece of waste can be labeled with multiple picking difficulties.

[0066] For example, images of glass bottles can be labeled with picking difficulty as 'Suction-type gripper - Difficulty' and 'Finger-type gripper - Normal'.

[0067] The artificial intelligence model learns waste images labeled with picking difficulty and analyzes waste images collected by the image collection unit (1100) to output picking difficulty according to the type of gripper (2100).

[0068] According to an embodiment of the present invention, an artificial intelligence model for learning waste images labeled with picking difficulty may utilize deep learning or a neural network. The neural network models may include, but are not limited to, various types of models such as Convolutional Neural Networks (CNN), Region with Convolutional Neural Network (R-CNN), Region Proposal Network (RPN), Recurrent Neural Network (RNN), Stacking-based Deep Neural Network (S-DNN), State-Space Dynamic Neural Network (S-SDNN), Deconvolution Network, Deep Belief Network (DBN), Restructured Boltzmann Machine (RBM), Fully Convolutional Network, Long Short-Term Memory Network (LSTM), and Classification Network, such as GoogleNet, AlexNet, and VGG Network.

[0069] FIG. 6 is a diagram showing a picking difficulty standard table according to an embodiment of the present invention. Referring to FIG. 6, the picking difficulty standard table may include multiple classes, each having its own standard.

[0070] Specifically, the multi-class may include a first class based on the type of gripper, a second class based on the shape of the waste, a third class based on the material composition of the waste, and a fourth class based on the contamination level of the waste.

[0071] For example, the first class can be determined according to the type of gripper equipped by the robot arm. If the robot arm (2000) is equipped with a suction type gripper (2100a) and a finger type gripper (2100b), the first class may include two criteria: suction and finger.

[0072] Class 2 can be determined based on the shape of the waste. It can be determined based on whether the waste is a bottle or a bowl, or whether it is in its original form or damaged.

[0073] Class 3 can be determined based on the material of the waste. It can be determined based on whether the waste is plastic or glass.

[0074] Class 4 can be determined based on the level of contamination of the waste. It can be classified as Clean if uncontaminated and Dirty if contaminated.

[0075] The picking difficulty can be set by combining the criteria of each class. For example, if the robot arm is equipped with a suction gripper, the waste is a damaged bottle, the material is plastic, and the contamination level is clean, the picking difficulty may be Normal.

[0076] The number of classes included in the picking difficulty standard table according to the embodiment of the present invention is not limited. It can be appropriately determined as needed. In addition, the criteria for each class are not limited. Appropriate criteria can be determined as needed. The number of criteria is also not limited.

[0077] The order of classes included in the picking difficulty standard table is not restricted. The order of each class may be changed, and the picking difficulty may be determined regardless of the class order.

[0078] According to an embodiment of the present invention, the picking difficulty determination unit (1300) may change the picking difficulty standard table if the task of gripping a target object by a designated robot arm fails according to the determined picking difficulty, and the task failure rate exceeds a preset standard.

[0079] Specifically, the picking difficulty determination unit (1300) determines the picking difficulty and the robot arm designation unit (1400) designates a robot arm to perform the task according to the picking difficulty, but the gripping may fail. The cause of the gripping failure may be that the picking difficulty standard table was not properly set.

[0080] Therefore, the picking difficulty judgment unit (1300) can reduce the work failure rate by changing the picking difficulty standard table when the work failure rate exceeds a preset standard.

[0081] For example, when the suction type gripper (2100a) grips a damaged plastic bowl, the picking difficulty may be Normal. However, the failure rate of the task of the suction type gripper (2100a) gripping a damaged plastic bowl may be 20% or higher, which is a preset standard. In this case, the picking difficulty determination unit (1300) may change the picking difficulty to Difficulty when the suction type gripper (2100a) grips a damaged plastic bowl.

[0082] The use of all examples or exemplary terms (e.g., etc.) in the present invention is merely for the purpose of describing the invention in detail, and the scope of the invention is not limited by such examples or exemplary terms unless limited by the claims. Furthermore, a person skilled in the art will understand that various modifications, combinations, and changes may be made according to design conditions and factors within the scope of the claims or equivalents to which they are added.

[0083] It will be obvious to those skilled in the art that the present invention described above is not limited to the aforementioned embodiments and attached drawings, and that various substitutions, modifications, and changes are possible within the scope of the technical concept of the present invention.

[0084] The waste sorting system according to an embodiment of the present invention can automatically determine the difficulty of the task of a robot arm grasping a target object—that is, the picking difficulty—based on the type and shape of the waste, according to the type of gripper. By assigning the gripper most suitable for the sorting task based on the calculated picking difficulty, the success rate of waste sorting can be increased and work efficiency maximized, thus enabling its widespread application in the intelligent automated waste processing and recycling industries.

Claims

1. An image collection unit that collects waste images of waste moving on a conveyor; A target object determination unit that determines the above waste into a target object to be classified and a non-target object not to be classified; and A waste sorting system reflecting picking difficulty, comprising a picking difficulty determination unit that determines the picking difficulty, which is the difficulty of the task of the robot arm grasping the target object, according to the type of gripper equipped by the robot arm.

2. In Paragraph 1, A waste sorting system reflecting picking difficulty, further comprising a robot arm control unit that designates a robot arm to perform a task of gripping the target object according to the determined picking difficulty.

3. In Paragraph 1, The above image acquisition unit is a waste classification system that reflects picking difficulty, comprising a vision sensor capable of acquiring the above waste image.

4. In Paragraph 1, A waste sorting system reflecting picking difficulty, wherein the gripper is one of a suction-type gripper that grips the target object using air suction force and a finger-type gripper that grips the target object using a finger structure.

5. In Paragraph 4, A waste sorting system reflecting picking difficulty, wherein the robot arm is equipped with at least one of the suction-type gripper and the finger-type gripper.

6. In Paragraph 1, The above-mentioned picking difficulty determination unit is a waste classification system reflecting picking difficulty, comprising an artificial intelligence model trained with data including waste images in which the picking difficulty is labeled according to a preset picking difficulty standard table.

7. In Paragraph 6, The above picking difficulty standard table is a waste classification system that reflects picking difficulty, including multiple classes, each class having its own standard.

8. In Paragraph 7, A waste classification system reflecting picking difficulty, comprising a first class based on the type of gripper, a second class based on the shape of the waste, a third class based on the material composition of the waste, and a fourth class based on the contamination level of the waste.

9. In Paragraph 6, A waste sorting system reflecting picking difficulty, wherein the above-described picking difficulty determination unit changes the above-described picking difficulty standard table when the task of gripping the target object by a designated robot arm fails according to the determined picking difficulty, and the task failure rate exceeds a preset standard.

10. In Paragraph 2, A waste sorting system reflecting picking difficulty, wherein the above-described robot arm control unit designates a gripper among the grippers equipped by the robot arm to perform the task of gripping the target object.

Images