System for Waste Sorting with Self-Diagnosis and Compensation of Anomaly
The waste classification system addresses inefficiencies in resource recovery facilities by self-diagnosing and correcting abnormalities, enhancing efficiency and processing speed through automated detection and correction of issues in waste sorting equipment.
Patent Information
- Application Number
- KR1020250186625
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2025-12-01
- Publication Date
- 2026-11-06
- Estimated Expiration
- 2045-12-01
AI Technical Summary
Resource recovery facilities face inefficiencies due to transfer failures caused by excessive waste input or malfunctions in waste sorting equipment, with conventional systems only responding after failures occur, leading to reduced efficiency.
A waste classification system that self-diagnoses and corrects abnormalities using sensor data and throughput data, incorporating an image collection unit, robot arm, sensor unit, and correction unit to detect and address issues such as contamination and coordinate system abnormalities, and performs correction operations like lens cleaning and coordinate system resets.
The system enables efficient operation of resource recovery facilities by rapidly responding to failures, increasing the picking success rate and improving waste processing speed through automated correction operations.
Smart Images

Figure 112025134839414-PAT00003_ABST
Abstract
Description
Technology Field
[0001] The present invention relates to a waste classification system that self-diagnoses and corrects causes of abnormalities, and more specifically, to a waste classification system that self-diagnoses and corrects causes of abnormalities capable of detecting abnormal conditions and self-diagnosing based on sensor data and waste throughput data within a resource recovery facility, and performing a correction operation. Background Technology
[0002] The amount of waste generated from households, factories, and other sources is rapidly increasing. To create a sustainable society, technologies are being researched and developed to recover reusable resources from this waste.
[0003] A waste sorting process is necessary to recover reusable resources. While human labor has traditionally been involved in manually sorting waste, waste sorting devices that eliminate the need for human intervention are being introduced recently.
[0004] The waste sorting device can identify waste using artificial intelligence and sort waste using various sorting means, such as robotic arms.
[0005] A Material Recovery Facility is a facility that separates, sorts, and recovers useful resources from waste to process them for recycling. Recently, there have been attempts to construct large-scale material recovery facilities by connecting multiple waste sorting units.
[0006] However, transfer failures frequently occur during the process of transporting waste within resource recovery facilities. These failures are caused by various factors, such as excessive waste input or malfunctions in waste sorting equipment.
[0007] Conventional technology had a problem where the efficiency of resource recovery facilities was reduced because it responded only after a transfer failure occurred. Prior art literature
[0008] Korean Patent Registration No. 10-2339655 (Registration Date, December 10, 2021) The problem to be solved
[0009] The technical problem of the present invention is conceived in this regard, and the objective of the present invention is to provide a waste classification system that can detect abnormal conditions based on sensor data regarding the facility environment and waste throughput data within a resource recovery facility, and self-diagnose and correct the causes of abnormalities based on the detected abnormal conditions to perform correction operations. means of solving the problem
[0010] A waste sorting system for self-diagnosing and correcting abnormal causes according to one embodiment for realizing the purpose of the present invention described above comprises: an image collection unit that collects images of said waste on a conveyor belt transporting said waste; a robot arm that sorts target objects from said waste on said conveyor belt; a sensor unit comprising at least one of a first sensor that measures contamination data of a space where said image collection unit is installed or a second sensor that measures driving environment data of said robot arm; an abnormality detection unit that detects an abnormal state of said image collection unit or said robot arm based on sensor data including at least one of said contamination data or said driving environment data and throughput data including a classification workload for said target objects; and a correction unit that directs a correction operation for said image collection unit or said robot arm according to the detected abnormal state.
[0011] In one embodiment of the present invention, the first sensor may include a dust sensor that measures the concentration of suspended particles in the air.
[0012] In one embodiment of the present invention, the second sensor may include a vibration sensor that detects vibrations applied to the robot arm or the image collection unit.
[0013] In one embodiment of the present invention, the throughput data may further include a picking success rate corresponding to the success rate of the picking operation of the robot arm on the target object.
[0014] In one embodiment of the present invention, the abnormal state may include a contamination state of the image collection unit.
[0015] In one embodiment of the present invention, an air jet spray nozzle may be further included to perform a correction operation that removes contamination on the lens surface in a non-contact manner by spraying compressed air to the image collection unit.
[0016] In one embodiment of the present invention, the abnormal state may include a coordinate system abnormal state caused by vibration.
[0017] In one embodiment of the present invention, the correction operation may include a predetermined coordinate system reset operation to synchronize the coordinate system between the image acquisition unit and the robot arm.
[0018] In one embodiment of the present invention, the anomaly detection unit may include an artificial intelligence model trained on sensor data in which the contamination state of the image collection unit or the anomaly state of the coordinate system is labeled.
[0019] In one embodiment of the present invention, the robot arm may include either a suction-type gripper utilizing air suction force or a finger-type gripper utilizing a finger structure. Effects of the invention
[0020] According to a waste classification system that self-diagnoses and corrects such causes of abnormality, the resource recovery facility can be operated efficiently by detecting abnormal conditions and automatically performing correction operations.
[0021] In addition, the present invention can rapidly respond to conveying failures, thereby increasing the picking success rate by the robot arm and improving the processing speed of waste.
[0022] In addition, other features and advantages of the present invention may be newly identified through the embodiments of the present invention. Brief explanation of the drawing
[0023] FIG. 1 is a drawing showing a resource recovery center composed of a waste classification device according to one embodiment of the present invention. FIG. 2 is a block diagram illustrating a waste classification system that self-diagnoses and corrects abnormal causes according to an embodiment of the present invention. Figure 3 is a conceptual diagram illustrating the structure of the waste sorting device of Figure 2. Figure 4 is a block diagram illustrating the waste sorting device of Figure 2. FIG. 5 is a drawing for explaining a robot arm according to one embodiment of the present invention. FIG. 6 is a drawing for explaining the types of grippers according to one embodiment of the present invention. Figure 7 is a block diagram for explaining the sensor part of Figure 4. Figure 8 is a block diagram illustrating the throughput calculation unit of Figure 2. FIG. 9 is a diagram to help understand the process of a throughput calculation unit calculating a waste throughput according to an embodiment of the present invention. Specific details for implementing the invention
[0024] It should be noted that in assigning reference numbers to the components of each drawing in this specification, identical components are assigned the same number whenever possible, even if they are shown in different drawings. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. As used in this specification, "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components in addition to the components mentioned.
[0025] The above objects, other objects, features, and advantages of the present invention will be easily understood through the following preferred embodiments associated with the accompanying drawings. However, the present invention is not limited to the embodiments described herein and may be embodied in other forms. Rather, the embodiments introduced herein are provided to ensure that the disclosed content is thorough and complete and to ensure that the spirit of the invention is sufficiently conveyed to a person skilled in the art.
[0026] Hereinafter, the present invention will be described in more detail with reference to the attached drawings.
[0027] FIG. 1 is a drawing showing a resource recovery center composed of a waste classification device according to one embodiment of the present invention.
[0028] Referring to FIG. 1, a resource recovery center may be composed of a plurality of waste sorting devices (1100). As shown in FIG. 1, a plurality of waste sorting devices (1100) may be connected in series. For example, a first waste sorting device (1100a), a second waste sorting device (1100b), a third waste sorting device (1100c), ..., an nth waste sorting device (1100n) may be connected in series. A plurality of waste sorting devices (1100) connected in series can rapidly sort waste in large quantities.
[0029] A resource recovery center can be constructed by arranging multiple lines of waste sorting devices (1100) connected in series. Each line can be configured in parallel, and depending on the conditions of the resource recovery center, the lines may be branched or merged.
[0030] The resource recovery center may include a number of waste sorting devices (1100). Therefore, if each waste sorting device (1100) is not properly managed, the overall efficiency of the resource recovery center may be significantly reduced.
[0031] FIG. 2 is a block diagram illustrating a waste classification system that self-diagnoses and corrects abnormal causes according to an embodiment of the present invention.
[0032] Referring further to FIG. 2, a waste classification system (1000) for self-diagnosing and correcting abnormal causes according to one embodiment of the present invention may include a waste classification device (1100), a throughput calculation unit (1200), an abnormality detection unit (1300), and a correction unit (1400). In one embodiment, the waste classification system (1000) for self-diagnosing and correcting abnormal causes may include at least one waste classification device (1100). For example, the waste classification system (1000) for self-diagnosing and correcting abnormal causes may be part of a system controlling the operation of a resource recovery center that includes a plurality of waste classification devices (1100).
[0033] Hereinafter, a waste classification device (1100) will be described with reference to FIGS. 3 to 7.
[0034] FIG. 3 is a conceptual diagram illustrating the structure of the waste sorting device of FIG. 2. FIG. 4 is a block diagram illustrating the waste sorting device of FIG. 2. FIG. 5 is a drawing illustrating a robot arm according to an embodiment of the present invention. FIG. 6 is a drawing illustrating a type of gripper according to an embodiment of the present invention. FIG. 7 is a block diagram illustrating the sensor unit of FIG. 4.
[0035] Referring to FIGS. 3 to 7, the waste sorting device (1100) can automatically sort waste (10) moving on a conveyor belt (2000). The waste sorting device (1100) can collect images of the waste (10), analyze the waste images to identify and classify target objects to be classified and non-target objects not to be classified.
[0036] When waste (10) is discharged onto the conveyor belt (2000), it can move in the direction (D1) in which the conveyor belt (2000) moves. Waste (10) can move onto the conveyor belt (2000) without being aligned or positioned. Target objects and non-target objects move while mixed randomly, and non-target objects may surround target objects, or non-target objects and target objects may overlap.
[0037] Each waste sorting device (1100) may use multiple robot arms (310, 320) to efficiently collect waste. Two or more robot arms (310, 320) may be placed adjacent to each other to efficiently utilize space.
[0038] A waste classification device (1100) according to one embodiment of the present invention may include an image collection unit (100), a target object identification unit (200), a robot arm (300), and a sensor unit (400).
[0039] The image collection unit (100) can collect images of waste moving on a conveyor belt (2000). The image collection unit (100) can collect images of waste and then preprocess the image data.
[0040] The waste images collected by the image collection unit (100) include 2D images and 3D images, and may include all images according to the type and function of the sensor.
[0041] The image acquisition unit (100) may include a vision sensor capable of acquiring 2D images and 3D images. For example, the image acquisition unit (100) may include a vision camera capable of capturing 2D images, and may include a Time of Flight (TOF) camera or a LiDAR camera capable of capturing 3D images. The type and function of the vision sensor are not limited and can be appropriately selected.
[0042] The image collection unit (100) may be installed at various locations according to the embodiment. For example, the image collection unit (100) may include a first image collection unit (110) installed at the beginning of the waste moving on the conveyor belt (2000), and second and third image collection units (120, 130) installed by being coupled to robot arms (310, 320).
[0043] Specifically, the image collection unit (100) may include a second image collection unit (120) installed on the first robot arm (310) and a third image collection unit (130) installed on the second robot arm (320), but is not limited thereto. In another embodiment, the image collection unit (100) may further include an image collection unit (not shown) installed between a plurality of robot arms on a conveyor belt (2000) or an image collection unit (not shown) installed at the rear end of the conveyor belt (2000), which is a point where waste is discharged.
[0044] The first image collection unit (110) can collect images of waste moving on a conveyor belt, and the second and third image collection units (120, 130) can collect images of target objects that can be classified and images of non-target objects that are not subject to classification within the activity radius of robot arms (310, 320) that classify target objects.
[0045] The image collection unit (100) may further include an air jet spray nozzle (111).
[0046] The air jet spray nozzle (111) can perform a correction operation to remove contamination on the lens surface in a non-contact manner by spraying compressed air to the first image collection unit (110) in accordance with the correction instructions of the correction unit (1400).
[0047] The air jet spray nozzle (111) can spray compressed air generated through a compressed air generating device (not shown), such as a compressor, onto the lens of the first image collecting unit (110) through the nozzle. For example, the air jet spray nozzle (111) can spray compressed air at an oblique angle of 20° to 60° with respect to the lens surface, but is not limited thereto. The shape of the discharge port of the air jet spray nozzle (111) can be implemented in various shapes such as a circle, a slit, or a venturi.
[0048] Although not shown, the image collection unit (100) may further include at least one air jet spray nozzle that performs a correction operation to remove contamination on the lens surface in a non-contact manner by spraying compressed air to the second and third image collection units (120, 130) in accordance with the correction instructions of the correction unit (1400). The air jet spray nozzle that sprays compressed air to the second and third image collection units (120, 130) is identical to the air jet spray nozzle (111) described above, except for the installation location, so a detailed description may be omitted.
[0049] The target object identification unit (200) can analyze waste images of waste (10) moving on a conveyor belt (2000). The target object identification unit (200) acquires waste images collected by the image collection unit (100) and can determine whether the acquired waste images are images of target objects to be classified using an artificial intelligence model.
[0050] The artificial intelligence model used by the target object discrimination unit (200) may be a deep learning or neural network. The neural network models may include various types of models such as Convolutional Neural Network (CNN), Region with Convolutional Neural Network (R-CNN), Region Proposal Network (RPN), Recurrent Neural Network (RNN), Stacking-based Deep Neural Network (S-DNN), State-Space Dynamic Neural Network (S-SDNN), Deconvolution Network, Deep Belief Network (DBN), Restricted Boltzmann Machine (RBM), Fully Convolutional Network, Long Short-Term Memory Network (LSTM), Classification Network, etc., such as GoogleNet, AlexNet, VGG Network, etc., but are not limited thereto.
[0051] The artificial intelligence model used by the target object identification unit (200) can learn waste images labeled according to a pre-set waste classification standard table.
[0052] A robot arm (300) according to one embodiment of the present invention may be positioned adjacent to each other on a conveyor belt (2000) to classify target objects. The robot arm (300) may include a plurality of robot arms. For example, the robot arm (300) may be composed of a first robot arm (310) and a second robot arm (320), but is not limited thereto and may be composed of a plurality of robot arms as needed.
[0053] According to one embodiment of the present invention, each robot arm (310, 320) may include a gripper (311, 321) that performs a gripping operation on a target object. The gripper (311, 321) may be formed to be detachable so that a gripper (311, 321) suitable for the purpose can be selected and coupled to the robot arms (310, 320), but is not limited thereto and the gripper and the robot arm may be formed as a single unit.
[0054] Each robot arm (310, 320) can be detached by selecting either a suction type gripper (a) that uses air suction force or a finger type gripper (b) that uses a finger structure as a gripper (311, 321).
[0055] A suction type gripper (a) is lightweight and may be advantageous for gripping flat objects. For example, a suction type gripper may be used when the proportion of waste brought into a resource recovery facility consists of PET bottles and plastic containers.
[0056] A finger-type gripper (b) may be advantageous for gripping objects that are heavy or do not have relatively flat parts. For example, a finger-type gripper may be used when the proportion of waste brought into a waste recovery facility consists of glass bottles and beverage cans.
[0057] Each robot arm (310, 320) may be equipped with a connecting means capable of detaching both a suction gripper and a finger gripper. For example, the robot arm may detach the suction gripper and the finger gripper using a rotary tab, and the suction gripper and the finger gripper may include a rotary tab of the same size.
[0058] The sensor unit (400) may include at least one of a first sensor that measures pollution level data of the space where the image collection unit (100) is installed, or a second sensor that measures operating environment data of the robot arm (300).
[0059] In one embodiment, the first sensor may include a dust sensor (410) that measures the concentration of suspended particles in the air. The second sensor may include a vibration sensor (420) that detects vibrations applied to the robot arm (300) or the image collection unit (100).
[0060] The sensor unit (400) may further include a temperature sensor (430) or a humidity sensor (440).
[0061] The dust sensor (410) can measure the concentration of suspended particles in the air in real time. The pollution data of the space where the image collection unit (100) and the robot arm (300) are installed may include the concentration of suspended particles measured by the dust sensor (410).
[0062] The dust sensor (410) can be any of the various types of dust sensors without limitation, such as a laser-based fine dust (PM2.5, PM10) sensor, a light scattering type, a light transmission type, or a capacitive detection type. For example, a light scattering type dust sensor calculates the dust concentration by detecting the degree to which light irradiated from a light-emitting part is scattered by particles in the air at a light-receiving part, a light transmission type dust sensor estimates the particle concentration by measuring the change in light transmittance, and a capacitive dust sensor can calculate the concentration of dust in the air by detecting the change in charge that occurs when particles collide.
[0063] The vibration sensor (420) can measure vibrations applied to the robot arm (300) or the image acquisition unit (100) in real time. The driving environment data of the image acquisition unit (100) and the robot arm (300) may include values corresponding to vibrations, such as displacement, velocity, acceleration, and frequency, measured by the vibration sensor (420).
[0064] The vibration sensor (420) can be any type of vibration sensor without limitation, such as a MEMS accelerometer, a natural vibration sensor, an optical vibration sensor, or a piezovoltage sensor.
[0065] For example, MEMS accelerometers can detect acceleration components by converting the displacement of a mass within a microelectromechanical system (MEMS) structure into an electrical signal. Natural vibration sensors can detect vibration frequencies by utilizing the natural vibration frequencies of a structure. Optical vibration sensors can measure the frequency, amplitude, and phase of vibrations by precisely measuring minute vibration displacements in a non-contact manner using laser interferometry or reflected light modulation principles. Piezovoltage sensors can calculate the intensity and frequency of vibrations by detecting voltage changes caused by vibration stress applied to a piezoelectric element.
[0066] The temperature sensor (430) can measure the temperature of the space where the image collection unit (100) and the robot arm (300) are installed. The humidity sensor (440) can measure the humidity of the space where the image collection unit (100) and the robot arm (300) are installed. Various types of sensors for measuring temperature and humidity can be applied to the temperature sensor (430) and the humidity sensor (440) without limitation.
[0067] The operating environment data may further include the temperature and humidity measured by the temperature sensor (430) and the humidity sensor (440).
[0068] FIG. 8 is a block diagram for explaining the throughput calculation unit of FIG. 2. FIG. 9 is a diagram to help understand the process of the throughput calculation unit according to an embodiment of the present invention calculating the waste throughput.
[0069] Referring to FIGS. 8 and 9, the throughput calculation unit (1200) can calculate the throughput for a target object for each waste classification device (1100) to generate and analyze throughput data.
[0070] In one embodiment, throughput data may include a classification workload for target objects. For example, throughput data may include a throughput per unit time for the task of classifying target objects by robot arms (310, 320). Since the throughput per hour may vary depending on the composition ratio of target objects fed into the conveyor belt (2000), throughput data may further include information on the composition ratio of target objects. For example, the composition ratio of target objects may be the composition ratio of objects to be classified, such as PET bottles, plastic containers, glass bottles, and beverage cans. The composition ratio of target objects may be calculated by analyzing a waste image of waste (10) moving on the conveyor belt (2000) by the target object identification unit (200).
[0071] Throughput data may further include picking success rate data corresponding to the success rate of the picking operation of the target object of the robot arms (310, 320). The picking success rate may vary depending on the type of gripper (311, 321) and the type of target object. Accordingly, the picking success rate data may be composed of picking success rate values associated with the type of gripper and the target object.
[0072] In contrast, the throughput calculation unit (1200) can analyze the distribution characteristics regarding the arrangement of waste (10) on the conveyor belt (2000) passing through the waste sorting device (1100). The distribution characteristics may include at least one of a blank area where waste (10) is not placed on the conveyor belt (2000) and the number of wastes.
[0073] Waste fed into the waste sorting device can be placed on a conveyor belt. If the amount of waste is small, empty areas on the conveyor belt where waste is not placed may occur, and if the amount of waste is large, the empty areas on the conveyor belt where waste is not placed can be reduced.
[0074] The amount of waste can be expressed in terms of the number of waste items. The number of waste items can be expressed as the total number, the number by type, or the number by category, and the criteria for counting the number of waste items can be appropriately selected.
[0075] The throughput calculation unit (1200) can calculate the waste throughput by recognizing an empty area on the conveyor belt where waste is not placed. In one embodiment, the throughput data may further include the waste throughput. For example, the throughput calculation unit (1200) can calculate that the waste throughput is 3 kg / min when the area of the empty area on the conveyor belt that has passed through the waste sorting device (1100) where waste is not placed is 30%.
[0076] The throughput calculation unit (1200) can calculate the amount of waste processed by counting the number of waste placed on the conveyor belt. For example, the throughput calculation unit (1200) can calculate that the amount of waste processed is 5 kg / min when the number of waste placed on the conveyor belt that has passed through the waste sorting device (1100) is 30.
[0077] The waste processing capacity may vary depending on the specifications and operating conditions of the input waste sorting device, the type and condition of the waste, and the scale and characteristics of the resource recovery center.
[0078] To calculate the waste processing amount, the processing amount calculation unit (1200) may include a vision sensor (1210), a first artificial intelligence model (1220), and a second artificial intelligence model (1230).
[0079] The vision sensor (1210) can acquire at least one of a first image, which is an image of waste at the point where waste is fed into the waste sorting device (1100), and a second image, which is an image of waste at the point where waste is discharged from the waste sorting device.
[0080] A vision sensor (1210) may be installed to acquire a first image, which is an image of waste at the point where waste is fed into the waste sorting device (1100). In one embodiment, the vision sensor (1210) may include a first image collection unit (110). The vision sensor (1210) may acquire the first image through the first image collection unit (110).
[0081] A vision sensor (1210) may be installed to acquire a second image, which is an image of waste at the point where waste is discharged into the waste sorting device (1100). The location of the vision sensor (1210) is not limited, and an appropriate location capable of acquiring the first image or the second image may be selected.
[0082] Accordingly, there may be at least one vision sensor (1210). For example, one vision sensor (1210) may be installed at the front end of the waste sorting device (1100), and another vision sensor (1210) may be installed at the rear end.
[0083] A vision sensor (1210) can acquire at least one of a first image or a second image, and a single vision sensor (1210) can acquire both the first image and the second image.
[0084] For example, a plurality of waste sorting devices (1100) may be arranged in series. In this case, a vision sensor (1210) may be installed between the first waste sorting device (1100a) and the second waste sorting device (1100b). This vision sensor (1210) can acquire a second image of the first waste sorting device (1100a) and a first image of the second waste sorting device (1100b). According to this example, the second image of the first waste sorting device (1100a) and the first image of the second waste sorting device (1100b) may be the same image.
[0085] As another example, a vision sensor (1210) may be rotatably installed in the center of the waste sorting device (1100). When waste is fed into the waste sorting device (1100), the rotatably installed vision sensor (1210) can acquire a first image. When the robot arm (1420) completes the sorting operation, the vision sensor (1210) rotates in the opposite direction to acquire a second image of the discharged waste.
[0086] The throughput calculation unit (1200) can compare the area of the blank region included in the first image and the second image using the learned first artificial intelligence model (1220), and calculate the amount of waste processed by the waste classification device (1100) based on the difference in the area of the blank region.
[0087] Specifically, the first artificial intelligence model (1220) can recognize the empty area of the conveyor belt where waste is not placed in the first image and the second image and calculate the area of the empty area. For example, as shown in FIG. 9 (a), the first artificial intelligence model (1220) can calculate that the area of the empty area where waste is not placed in the first image is 2% of the total area of the first image. As shown in FIG. 9 (b), the first artificial intelligence model (1220) can calculate that the area of the empty area where waste is not placed in the second image is 75% of the total area of the second image.
[0088] The first artificial intelligence model (1220) can calculate the amount of waste processed by the waste sorting device (1100) based on the difference in the area of the blank region. For example, as shown in (c) of FIG. 9, the first artificial intelligence model can calculate that the area of the blank region included in the first image is 25% of the area of the first image. As shown in (d) of FIG. 9, the first artificial intelligence model (1220) can calculate that the area of the blank region included in the second image is 70% of the area of the second image. The first artificial intelligence model (1220) can calculate that 4.5 kg of waste has been processed, which corresponds to 45% of the difference between the area of the blank region of the first image and the area of the blank region of the second image.
[0089] According to another embodiment of the present invention, the throughput calculation unit (1200) can compare the number of waste items included in the first image and the second image using a learned first artificial intelligence model (1220), and calculate the amount of waste processed by the waste classification device (1100) based on the difference in the number of waste items.
[0090] Specifically, the first artificial intelligence model (1220) can recognize waste in the first image and the second image and count the number of waste items. For example, as shown in FIG. 9 (a), the first artificial intelligence model (1220) can count the number of waste items in the first image as 30. As shown in FIG. 9 (b), the first artificial intelligence model (1220) can count the number of waste items in the second image as 8.
[0091] The first artificial intelligence model (1220) can calculate the amount of waste processed by the waste sorting device (1100) based on the difference in the number of waste items. For example, as shown in (c) of FIG. 9, the first artificial intelligence model can calculate that the number of waste items included in the first image is 25. As shown in (d) of FIG. 9, the first artificial intelligence model (1220) can calculate that the number of waste items included in the second image is 15. The first artificial intelligence model (1220) can calculate that 2 kg of waste, corresponding to the difference of 10 between the number of waste items in the first image and the number of waste items in the second image, has been processed.
[0092] According to an embodiment of the present invention, the first artificial intelligence model (1220) may be a model that has been pre-trained with data including the area of the blank region included in the first image and the second image, the number of wastes, and the amount of waste processed.
[0093] Specifically, the first artificial intelligence model (1220) can learn input data including the area of the blank area and the number of waste, and output data including the waste processing volume. For example, in addition to the area of the blank area and the number of waste, the input data may include the weight of each type of waste, information on resource recovery centers, information on waste sorting devices, etc. The output data may include the processing volume of each type of waste, etc. The input data and output data may include other data, and the types of data are not limited.
[0094] According to an embodiment of the present invention, the second artificial intelligence model (1230) may be a model that has been pre-trained with data including the area of the blank region included in the first image, the number of wastes, and the amount of waste processed.
[0095] Specifically, the second artificial intelligence model (1230) can learn input data including the area of the blank area and the number of waste, and output data including the waste processing volume. For example, in addition to the area of the blank area and the number of waste, the input data may include the weight of each type of waste, information on resource recovery centers, information on waste sorting devices, waste processing volume, etc. The output data may include the processing volume of each type of waste, etc. The input data and output data may include other data, and the types of data are not limited.
[0096] The throughput calculation unit (1200) can predict the waste processing volume of the waste sorting device (1100) based on the area of the blank region included in the first image through the learned second artificial intelligence model (1230).
[0097] Specifically, the second artificial intelligence model (1230) can recognize the empty area of the conveyor belt where waste is not placed in the first image and calculate the area of the empty area. The second artificial intelligence model (1230) can calculate the amount of waste input based on the area of the empty area included in the first image and predict the amount of waste that the waste sorting device (1100) can process.
[0098] For example, the second artificial intelligence model (1230) can calculate that the area of the blank region included in the first image is 5% of the area of the first image. The second artificial intelligence model (1230) can calculate that 9.5 kg of waste corresponding to 5% of the area of the blank region of the first image has been input. The second artificial intelligence model (1230) can predict that the waste sorting device (1100) can process 4 kg of the 9.5 kg of waste input.
[0099] According to another embodiment of the present invention, the throughput calculation unit (1200) can predict the waste throughput of the waste classification device (1100) based on the number of waste items included in the first image through a learned second artificial intelligence model (1230).
[0100] Specifically, the second artificial intelligence model (1230) can recognize waste in the first image and count the number of waste items. The second artificial intelligence model (1230) can calculate the amount of waste to be processed based on the number of waste items included in the first image and predict the amount of waste to be processed by the waste sorting device (1100).
[0101] For example, the second artificial intelligence model (1230) can calculate that the number of waste items included in the first image is 50. The second artificial intelligence model (1230) can calculate that 9.5 kg of waste corresponding to the number of waste items in the first image, which is 50, has been input. The second artificial intelligence model (1230) can predict that the waste sorting device (1100) can process 4 kg of the input 9.5 kg of waste.
[0102] The anomaly detection unit (1300) can detect an abnormal state of the image collection unit (100) or robot arm (300) for each waste classification device (1100) based on sensor data and throughput data including at least one of pollution level data or operating environment data.
[0103] In one embodiment, the anomaly detection unit (1300) can detect an abnormal state of the image collection unit (100) or the robot arm (300) based on sensor data and throughput data. The abnormal state detected by the anomaly detection unit (1300) may include at least one of the contamination state of the image collection unit (100) and a coordinate system abnormal state caused by vibration.
[0104] For example, the anomaly detection unit (1300) can detect an abnormal state of the image collection unit (100) or the robot arm (300) based on the hourly throughput of the classification work for the target object and the picking success rate.
[0105] Specifically, if the picking success rate remains within the normal range but the hourly throughput decreases, it can be considered that an abnormality has occurred in the recognition of the target object. Therefore, if the picking success rate remains within the normal range but the hourly throughput decreases, camera contamination should be suspected.
[0106] In contrast, if the picking success rate and the hourly throughput decrease simultaneously, it can be seen that an abnormality has occurred in the picking operation of the target object. Therefore, if the picking success rate and the hourly throughput decrease simultaneously, an abnormality in the coordinate system between the image collection unit (100) and the robot arm (300) can be suspected.
[0107] To detect an abnormal state of an image collection unit (100) or a robot arm (300), the abnormality detection unit (1300) may include an artificial intelligence model (hereinafter, an abnormality detection model) trained with training data that includes sensor data and throughput data in which at least one abnormal state is labeled.
[0108] For example, the anomaly detection unit (1300) may include an anomaly detection model trained with sensor data and throughput data labeled as a normal state, a contamination state of the image collection unit (100), or a coordinate system anomaly state due to vibration, but is not limited thereto and may be trained with sensor data and throughput data labeled as various anomaly states.
[0109] According to an embodiment of the present invention, an artificial intelligence model that learns sensor data and throughput data labeled with abnormal states can be configured to learn normal and abnormal states by inputting labeled sensor data (e.g., multivariate sensor values such as dust concentration, vibration value, temperature, humidity, etc.) and data related to throughput.
[0110] The artificial intelligence model for learning the anomaly detection model, the first artificial intelligence model (1220), and the second artificial intelligence model (1230) is not limited to a specific structure, and various deep learning or neural network-based models may be applied. For example, time series models such as RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory) network, and GRU (Gated Recurrent Unit) to reflect temporal correlations, CNN (Convolutional Neural Network) or Fully Convolutional Network for extracting features of multivariate sensor patterns, and CNN-RNN hybrid models for learning time series and spatial features simultaneously may be used. In addition, probabilistic feature-based models such as DBN (Deep Belief Network) and RBM (Restricted Boltzmann Machine), or state-space based S-DNN (Stacking-based Deep Neural Network) and S-SDNN (State-Space Dynamic Neural Network) may also be used without restriction to learn patterns of anomalies. Furthermore, when sensor data is converted into a two-dimensional data form such as a spectrogram, existing image recognition models such as GoogleNet, AlexNet, and VGG Network can also be applied.
[0111] As such, the anomaly detection model of the present invention can be trained by selectively applying various types of neural network models based on labeled sensor data and throughput data, and is not limited to a specific model structure.
[0112] The correction unit (1400) can instruct the image collection unit (100) or the robot arm (300) to perform a correction operation according to the detected abnormal state.
[0113] When the detected abnormal state corresponds to a contamination state of the image collection unit (100), the correction unit (1400) may instruct the air jet spray nozzle (111) to spray compressed air toward the image collection unit (100) to remove contamination from the lens surface in a non-contact manner.
[0114] The correction unit (1400) may instruct a correction operation including a predetermined coordinate system reset operation to synchronize the coordinate system between the image collection unit (100) and the robot arm (300) when the detected abnormal state corresponds to a coordinate system abnormal state caused by vibration.
[0115] The coordinate system reset operation can be configured as a predetermined series of processes to synchronize the coordinate system between the image acquisition unit (100) and the robot arm (300) in software, and can be configured in various ways.
[0116] Meanwhile, the correction unit (1400) receives the predicted waste processing amount from the processing amount calculation unit (1200) and can determine the waste classification device (1100) to be operated by considering the predicted waste processing amount and the preset processing speed standard together.
[0117] In order to efficiently operate a resource recovery center composed of multiple waste sorting devices (1100), it may be necessary to operate all waste sorting devices (1100) or only a portion of the waste sorting devices (1100) depending on the situation. In such cases, the correction unit (1400) can determine whether to operate all waste sorting devices (1100) or only a portion of the waste sorting devices (1100) based on the predicted waste processing volume and a preset waste processing volume standard.
[0118] For example, the throughput calculation unit (1200) can predict the waste processing amount of each waste sorting device (1100) to be 7 kg / min. The preset waste processing amount standard may be 5 kg / min. In this case, the correction unit (1400) can operate 5 of the total 10 waste sorting devices in the resource recovery center.
[0119] The correction unit (1400) can accumulate the amount of waste processed by the waste sorting device (1100) during a preset period and determine the waste sorting device to be operated by taking into account the accumulated amount of waste processed.
[0120] In a resource recovery center composed of multiple waste sorting devices (1100), if only some of the waste sorting devices (1100) are operated according to the amount of waste processed, the level of aging may vary for each waste sorting device (1100). According to an embodiment of the present invention, by varying the waste sorting devices (1100) to be operated according to the accumulated amount of waste processed, it is possible to prevent only some of the waste sorting devices (1100) from becoming aging.
[0121] For example, the preset period may be one month. During the past month, the amount of waste processed by the first waste sorting device (1100a) may be 5000 kg, the amount of waste processed by the second waste sorting device (1100b) may be 3000 kg, and the amount of waste processed by the third waste sorting device (1100c) may be 2500 kg. When the correction unit (1400) needs to operate some of the waste sorting devices (1100), it may stop the operation of the first waste sorting device (1100a), which has the largest accumulated amount of waste processed, and operate the remaining waste sorting devices.
[0122] The correction unit (1400) can determine the operating conditions of the waste sorting device (1100) to be operated. Specifically, the correction unit (1400) can increase or decrease the operating conditions of the waste sorting device (1100) if the predicted waste processing amount differs from the preset waste processing amount standard.
[0123] For example, the predicted waste processing volume may be 3 kg / min and the preset waste processing volume standard may be 5 kg / min. The correction unit (1400) may decide to operate the entire waste sorting device (1100) within the resource recovery center and, at the same time, increase the operating speed of the conveyor by 20% compared to the existing speed.
[0124] According to the waste classification system for self-diagnosing and correcting abnormal causes configured as described above, a waste classification system can be provided that detects and corrects abnormal causes through abnormal detection, thereby enabling efficient operation of the waste classification device.
[0125] Although the invention has been described with reference to the above embodiments, those skilled in the art will understand that various modifications and changes can be made to the invention without departing from the spirit and scope of the invention as described in the following claims. Explanation of the symbols
[0126] 100: Image collection unit 200: Target object identification unit 300: Robot arm 400: Sensor unit 410: Dust sensor 420: Vibration sensor 430: Temperature sensor 440: Humidity sensor 1100: Waste Classification System 1200: Throughput Calculation Unit 1300: Anomaly detection unit 1400: Correction unit
Claims
Claim 1 A waste classification system that self-diagnoses and corrects the cause of an abnormality, comprising: an image collection unit that collects images of waste on a conveyor belt transporting waste; a robot arm that classifies target objects from the waste on the conveyor belt; a sensor unit that includes at least one of a first sensor that measures contamination data of a space where the image collection unit is installed or a second sensor that measures driving environment data of the robot arm; an abnormality detection unit that detects an abnormal state of the image collection unit or the robot arm based on sensor data including at least one of the contamination data or driving environment data and throughput data including a classification workload for the target objects; and a correction unit that directs a correction operation for the image collection unit or the robot arm according to the detected abnormal state, wherein the abnormal state includes a contamination state of the image collection unit and a coordinate system abnormal state caused by vibration, and the correction operation includes a predetermined coordinate system reset operation to synchronize the coordinate system between the image collection unit and the robot arm. Claim 2 A waste classification system for self-diagnosing and correcting abnormal causes, characterized in that, in claim 1, the first sensor includes a dust sensor that measures the concentration of suspended matter in the air. Claim 3 A waste sorting system for self-diagnosing and correcting abnormal causes, characterized in that, in claim 1, the second sensor includes a vibration sensor that detects vibrations applied to the robot arm or the image collection unit. Claim 4 A waste sorting system for self-diagnosing and correcting abnormal causes, characterized in that, in claim 1, the throughput data further includes a picking success rate corresponding to the success rate of the picking operation of the robot arm on the target object. Claim 5 delete Claim 6 A waste classification system for self-diagnosing and correcting abnormal causes, characterized in that, in claim 1, it further includes an air jet spray nozzle that performs a correction operation to remove contamination from the lens surface in a non-contact manner by spraying compressed air into the image collection unit. Claim 7 delete Claim 8 delete Claim 9 A waste classification system for self-diagnosing and correcting causes of abnormalities, wherein, in claim 1, the abnormality detection unit includes an artificial intelligence model trained on sensor data labeled with the contamination state of the image collection unit or the abnormal state of the coordinate system. Claim 10 A waste sorting system for self-diagnosing and correcting the cause of an abnormality, wherein, in claim 1, the robot arm comprises either a suction-type gripper utilizing air suction force or a finger-type gripper utilizing a finger structure.
Citation Information
Patent Citations
Robot control device and robot system
JP7460800B2
Industrial CCTV forming air curtain with improved dust barrier properties
KR1020240002576A
Waste sorting machine operating system
KR102889722B1
System and method for automated cleaning
US20220194328A1
Waste sorting robot with throw sensor for determining position of waste object
WO2022090627A1
