Target object classification system using multi-class classification reference table

Abstract

The present invention provides a target object classification system using a multi-class classification reference table, the system comprising: a data generation unit for generating, as data, a waste image obtained from a vision camera for collecting an image of a waste moving on a conveyor belt; a target object determination unit for determining whether the waste image is a target object to be classified, by using an artificial intelligence model, on the basis of a waste classification reference table having multiple classes; and a robot designation unit for designating a robot to classify the target object, by using the waste classification reference table.

Description

Object classification system using multi-class classification criteria

[0001] The present invention relates to a target object classification system for classifying waste, and more specifically, to a target object classification system using a multi-class classification criterion for classifying target objects using a multi-class classification criterion.

[0002] A large amount of waste is produced every day from homes, factories, restaurants, etc., and among general waste, recyclable waste, and food waste, many people separate and throw away food waste.

[0003] However, recyclable waste is still being discharged mixed with general waste, and manpower is being directly invested in separating it.

[0004] Moreover, for companies that use PET or glass bottles, the problem of recyclable waste being disposed of as general waste without being properly separated is even more serious, making it necessary to sort recyclable waste.

[0005] To solve these problems, various inventions are being made, and various methods, devices, and systems for separating waste are being applied for and registered as patents.

[0006] The inventor of the present invention has previously disclosed a method for determining waste to be classified based on classification information including information on the type of waste, contamination level, color, and presence or absence of a cap and label in Patent Publication No. 10-2222177, "Recycling classification system using AI."

[0007] However, these classification criteria were difficult to specify each target object because the types and conditions of waste were so diverse, and there were limitations in directly labeling and updating the classification information for all of these wastes.

[0008] Additionally, there was a problem that the efficiency of waste sorting was low because all waste to be sorted was assigned to the same robot.

[0009] The technical problem to be achieved by the present invention is to provide a target object classification system using a multi-class criteria table that allows a user to easily designate a target object and is easy to upgrade.

[0010] In addition, another technical task to be achieved by the present invention is to provide a target object classification system using a multi-class reference table that can improve waste classification efficiency through matching combinations of target objects to be classified with multiple robots.

[0011] In order to solve such a problem, a target object classification system using a multi-class classification criteria table according to an embodiment of the present invention includes a data generation unit that generates data from a waste image obtained from a vision camera that collects an image of waste moving on a conveyor belt; a target object determination unit that determines, using an artificial intelligence model, whether the waste image is a target object to be classified based on a waste classification criteria table having multiple classes; and a robot designation unit that designates a robot to classify the target object using the waste classification criteria table.

[0012] The above artificial intelligence model can be trained using waste images labeled according to a preset waste classification standard as learning data.

[0013] The above waste classification criteria table may include multiple classes, each of which has its own classification criteria.

[0014] The first class of the above multi-class may include a category based on the material composition of the waste, a second class may include a category based on the condition of the waste (whether it is damaged or not), a third class may include a category based on the use of the waste, a fourth class may include a category based on the color of the waste, and a fifth class may include waste not included in the first class to the fourth class.

[0015] The above target object determination unit can determine whether or not the object is the target object based on the class of the waste classification criteria selected by the user.

[0016] The waste classification criteria generation unit may further include a waste classification criteria generation unit that adds a new classification attribute to the above waste classification criteria table based on the new classification attribute.

[0017] The above waste classification criteria generation unit can derive new classification attributes from waste images classified into the fifth class using an artificial intelligence model or upgrade the waste classification criteria table using new classification attributes input by a user.

[0018] The method may further include an artificial intelligence model learning unit that automatically labels new classification attributes added to the waste images classified into the fifth class and retrains the artificial intelligence model.

[0019] In the case where multiple robots are used, the robot designation unit can match the target object and the robots with each other by considering the type and number of classes selected by the user and the number of the robots.

[0020] The above robot designation unit can change the matching between the target object and the robot when the target object classification failure rate according to the matching between the target object and the robot is below a preset standard.

[0021] In addition to the technical problems of the present invention mentioned above, other features and advantages of the present invention are described below or may be clearly understood by a person skilled in the art to which the present invention pertains from such description and explanation.

[0022] According to the present invention as described above, the following effects are achieved.

[0023] The present invention can automatically derive new classification attributes and upgrade a multi-class classification criteria table without requiring the user to individually find new classification criteria and compare them with existing classification criteria tables to upgrade the multi-class classification criteria table.

[0024] The present invention can designate target objects that each robot can classify to achieve optimal classification efficiency by considering the number of target objects to be classified and the number of robots.

[0025] In addition, other features and advantages of the present invention may be newly discovered through the embodiments of the present invention.

[0026] FIG. 1 is a schematic diagram of a waste classification system using an artificial intelligence-based robot to which a target object classification system using a multi-class reference table is applied according to one embodiment of the present invention.

[0027] FIG. 2 is a schematic diagram of a target object classification system using a multi-class criteria table according to one embodiment of the present invention.

[0028] FIG. 3 is a drawing illustrating a multi-class criteria table according to one embodiment of the present invention.

[0029] FIG. 4 is a diagram for explaining a user selecting a target object using a multi-class criteria table according to one embodiment of the present invention.

[0030] FIG. 5 is a drawing for explaining how a waste classification criterion generation unit according to one embodiment of the present invention adds a new classification attribute.

[0031] FIG. 6 is a schematic diagram of a waste classification system using an artificial intelligence-based robot to which multiple robots are applied according to an embodiment of the present invention.

[0032] When adding reference numbers to components of each drawing in this specification, it should be noted that identical components are given the same numbers as much as possible even if they are shown in different drawings.

[0033] Meanwhile, the meanings of the terms described in this specification should be understood as follows.

[0034] Singular expressions should be understood to include plural expressions unless the context clearly defines otherwise, and terms such as “first”, “second”, etc. are intended to distinguish one component from another, and the scope of rights should not be limited by these terms.

[0035] The terms "include" or "have" should be understood as not excluding in advance the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0036] The term "at least one" should be understood to include all possible combinations of one or more associated items. For example, "at least one of the first, second, and third items" means any combination of items that can be represented by two or more of the first, second, and third items, as well as each of the first, second, and third items.

[0037] As used herein, the term "waste" includes plastics, PET bottles, glass bottles, glass, paper, Styrofoam, general waste, and industrial waste.

[0038] The target object used in this specification refers to an object to be classified among various types of objects such as waste.

[0039] Hereinafter, preferred embodiments of the present invention designed to solve the above problems will be described in detail with reference to the attached drawings.

[0040] FIG. 1 is a schematic diagram of a waste classification system using an artificial intelligence-based robot to which a target object classification system using a multi-class reference table is applied according to one embodiment of the present invention.

[0041] Referring to FIG. 1, a waste classification system using an artificial intelligence-based robot may include a target object classification system (1000) using a multi-class reference table according to the present invention, a vision camera (2000), and a robot (3000).

[0042] Objects (10), such as waste moving on a conveyor belt (4000), are collected through a vision camera (2000), and the target objects to be classified are selected through a target object classification system (1000) using a multi-class reference table according to the present invention, and then the target objects can be classified through a robot (3000).

[0043] The vision camera (2000) can collect an image of an object (10) and then transmit the image data to a target object classification system (1000) using a multi-class criteria table.

[0044] The vision camera (2000) may be installed in various locations depending on the embodiment. For example, a first vision camera (2100) may be installed at the beginning of the movement of waste on a conveyor belt (4000), and a second vision camera (2200) may be installed coupled to a robot (3000).

[0045] According to an embodiment, a target object classification system (1000) using a multi-class reference table for image data according to one embodiment of the present invention can select a target object through object (10) information collected through a first vision camera (2100), and designate a target object to be classified by a robot (3000) having a second vision camera (2200) installed through object (10) information collected through a second vision camera (2200).

[0046] In addition to the vision camera (2000), various sensors can be used as equipment for collecting images of an object (10) used in the present invention. For example, an optical sensor, an infrared sensor, an ultrasonic sensor, a metal detector, an x-ray sensor, a radar sensor, etc. can be used.

[0047] The robot (3000) is a robot for classifying target objects moving on a conveyor belt, and not only suction-type robotic arms but also gripper-type robotic arms with various grip forms can be used.

[0048] FIG. 2 is a schematic diagram of a target object classification system using a multi-class criteria table according to an embodiment of the present invention, FIG. 3 is a diagram explaining a multi-class criteria table according to an embodiment of the present invention, FIG. 4 is a diagram explaining a user selecting a target object using a multi-class criteria table according to an embodiment of the present invention, FIG. 5 is a diagram explaining a waste classification criteria generation unit adding a new classification attribute according to an embodiment of the present invention, and FIG. 6 is a schematic diagram of a waste classification system using an artificial intelligence-based robot arm to which a plurality of robots are applied according to an embodiment of the present invention.

[0049] Referring to FIGS. 1 and 2, a target object classification system (1000) using a multi-class criteria table according to the present invention includes a data generation unit (100), a target object determination unit (200), a robot designation unit (300), a waste classification criteria generation unit (400), and an artificial intelligence model learning unit (500).

[0050] The data generation unit (100) can generate data from waste information obtained from a vision camera (2000) that collects information on waste moving on a conveyor belt (4000).

[0051] The data generation unit (100) can perform data preprocessing on waste image data through color space correction, resolution correction, bit depth correction, density and transmittance correction.

[0052] The target object determination unit (200) can determine whether the waste image is a target object to be classified using an artificial intelligence model based on a waste classification criteria table having multiple classes.

[0053] The artificial intelligence model used to determine whether the target object is a target object may utilize deep learning or a neural network. Neural network models may include, but are not limited to, various types of models such as CNN (Convolution Neural Network), R-CNN (Region with Convolution Neural Network), RPN (Region Proposal Network), RNN (Recurrent Neural Network), S-DNN (Stacking-based deep Neural Network), S-SDNN (State-Space Dynamic Neural Network), Deconvolution Network, DBN (Deep Belief Network), RBM (Restrcted Boltzman Machine), Fully Convolutional Network, LSTM (Long Short-Term Memory) Network, Classification Network, etc.

[0054] Referring to FIG. 3, the waste classification criteria table according to the present invention may include multiple classes, each having its own classification criteria.

[0055] For example, the first class (Class 1) of a multi-class system may be a category based on the material composition of the waste, the second class (Class 2) may be a category based on the condition (whether damaged or not) of the waste, the third class (Class 3) may be a category based on the use of the waste, and the fourth class (Class 4) may be a category based on the color of the waste. The fifth class (Class 5) may be a category that includes waste that is not included in any of the first to fourth classes.

[0056] The target object determination unit (200) can determine whether or not it is a target object based on the class of the waste classification criteria selected by the user.

[0057] Users can freely select (Class 1, Class 2, Class 3, Class 4).

[0058] For example, referring to FIG. 4, a user can select (PP, damaged, yogurt, white), and the target object determination unit (200) can determine the target object using artificial intelligence based on a multi-class classification criterion selected by the user among images of waste moving on a conveyor belt.

[0059] Users can create multiple multi-class classifiers, as well as various class combinations. For example, a user can create a multi-class classifier {(PP, damaged, yogurt, white), (PET, none, none, black)}. Here, none means that no items in that class are selected, meaning that the classifier for that class is not considered.

[0060] The waste classification criteria generation unit (400) can add new classification attributes to the waste classification criteria table.

[0061] The waste classification criteria generation unit (400) can derive new classification attributes from waste images classified into the fifth class using an artificial intelligence model or upgrade the waste classification criteria table using new classification attributes input by the user.

[0062] In the past, in order to upgrade a waste classification standard, there was a difficulty in that the user had to individually specify the types and conditions of waste that were not in the existing waste classification standard. However, in the present invention, by utilizing waste images classified into the fifth class, new classification attributes can be automatically derived and upgraded through an artificial intelligence model.

[0063] The artificial intelligence model used at this time can analyze waste images classified into the fifth class and derive classification attributes corresponding to the new class.

[0064] For example, by extracting features from images of waste classified into the fifth class using a convolutional neural network (CNN), new classification attributes can be derived to classify images based on these features.

[0065] Here, the new classification attribute may be a new item of an existing class, or it may be a criterion for specifying a new class.

[0066] The artificial intelligence model learning unit (500) automatically labels new classification attributes added to waste images classified into the fifth class, and retrains the artificial intelligence model using the labeled waste images as training data to improve the performance of the target object determination unit.

[0067] Referring to FIG. 5, the waste classification criterion generation unit (400) can extract a whiskey bottle from among the waste classified as a fifth class (Class 3) because it is a glass bottle that is not included in the third class (Class 3) and add it as a new third class item (new classification attribute), and the artificial intelligence model learning unit (500) can automatically label the whiskey bottle as a new third class item and retrain the artificial intelligence model based on the labeled data.

[0068] In this way, the target object classification system using a multi-class classification criteria table according to one embodiment of the present invention can automatically derive new classification attributes and upgrade the multi-class classification criteria table without requiring the user to individually find new classification criteria and compare them with the existing classification criteria table to upgrade the multi-class classification criteria table.

[0069] The robot designation unit (300) can designate a robot to classify a target object using a multi-class waste classification criteria table.

[0070] In cases where multiple robots are used, the robot designation unit (300) can match the target object and the robots with each other by considering the type and number of classes selected by the user and the number of robots.

[0071] For example, a user can create target objects to be classified using multi-class classification criteria, such as {(PP, damaged, yogut, white), (PET, none, none, black)}, (PE, damaged, Drink, black)}, (Glass, original, soju, clear)}, (PET, original, Drink, clear)}, (PS, original, Sauce, white)}.

[0072] Referring to FIG. 6, the robot designation unit (300) can match a robot to classify each target object by considering the type and number of classes and the number of robots among the target objects selected by the user. For example, based on class 1, {(PET, none, none, black)}, (PET, original, Drink, clear)} can be designated to be classified by the first robot (2100), and {(PP, damaged, yogurt, white), (PE, damaged, Drink, black)}, (Glass, original, soju, clear)}, (PS, original, Sauce, white)} can be designated to be classified by the second robot (2200).

[0073] The robot designation unit (300) can change the matching between the target object and the robot when the target object classification failure rate according to the matching between the target object and the robot falls below a preset standard.

[0074] For example, based on the class 1 described above, {(PET, none, none, black)}, (PET, original, Drink, clear)} are matched with the first robot (2100), and {(PP, damaged, yogut, white), (PE, damaged, Drink, black)}, (Glass, original, soju, clear)}, (PS, original, Sauce, white)} are matched with the second robot (2200), and then, if the target object classification failure rate becomes lower than a preset standard, the matching can be changed based on class 2. In another embodiment, if there are 6 target object classifications selected by the user and 2 robots, each robot can divide and match 3 target object classifications.

[0075] In this way, the target object classification system using the multi-class classification criteria according to the present invention can designate the target objects that each robot can classify to have optimal classification efficiency by considering the number of target objects and robots that the user wants to classify.

[0076] The systems or devices described above may be implemented as hardware components, or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose computers or special-purpose computers, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing instructions and responding to them. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing device is sometimes described as being used alone; however, one of ordinary skill in the art will recognize that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing unit may include multiple processors, or a processor and a controller. Other processing configurations, such as parallel processors, are also possible.

[0077] Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing device to perform a desired operation or may independently or collectively command the processing device. The software and / or data may be embodied in any type of machine, component, physical device, virtual equipment, computer storage medium, or device for interpretation by the processing device or for providing instructions or data to the processing device. The software may be distributed over networked computer systems and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable recording media.

[0078] The method according to the embodiment may be implemented in the form of program commands that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., singly or in combination. The medium may continuously store a computer-executable program, or may temporarily store it for execution or download. In addition, the medium may be various recording means or storage means in the form of a single or multiple hardware combinations, and is not limited to a medium directly connected to a computer system, but may also be distributed over a network. Examples of the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and those configured to store program commands, including ROM, RAM, and flash memory. In addition, examples of other media may include recording media or storage media managed by app stores that distribute applications, sites that supply or distribute various software, servers, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0079] It will be apparent to those skilled in the art that the present invention described above is not limited to the above-described embodiments and the attached drawings, and that various substitutions, modifications, and changes are possible within the scope that does not depart from the technical spirit of the present invention.

Claims

1. A data generation unit that generates data from the waste image obtained from a vision camera that collects images of waste moving on a conveyor belt; A target object determination unit that determines whether the waste image is a target object to be classified using an artificial intelligence model based on a waste classification criteria table having multiple classes; and A target object classification system using a multi-class classification criteria table, including a robot designation unit that designates a robot to classify the target object using the above waste classification criteria table.

2. In paragraph 1, A target object classification system using a multi-class classification criteria table, characterized in that the above artificial intelligence model is learned using waste images labeled according to a preset waste classification criteria table as learning data.

3. In paragraph 2, A target object classification system using a multi-class classification criteria table, characterized in that the above waste classification criteria table includes multiple classes, each of which has its own classification criteria.

4. In paragraph 3, A target object classification system using a multi-class classification criteria table, characterized in that the first class of the above multi-class is a category according to the material composition of the waste, the second class is a category according to the condition (whether damaged) of the waste, the third class is a category according to the use of the waste, the fourth class is a category based on the color of the waste, and the fifth class includes waste not included in the first class to the fourth class.

5. In paragraph 4, A target object classification system using a multi-class classification criterion table, characterized in that the target object determination unit determines whether the target object is the target object based on the class of the waste classification criterion table selected by the user.

6. In paragraph 5, A target object classification system using a multi-class classification criteria table, further comprising a waste classification criteria generation unit that adds a new classification attribute to the above waste classification criteria table based on a new classification attribute.

7. In paragraph 6, A target object classification system using a multi-class classification criterion table, characterized in that the waste classification criterion generation unit derives new classification attributes from waste images classified into the fifth class using an artificial intelligence model or upgrades the waste classification criterion table using new classification attributes input by a user.

8. In paragraph 7, A target object classification system using a multi-class classification criteria table, further comprising an artificial intelligence model learning unit that automatically labels new classification attributes added to waste images classified into the fifth class and retrains the artificial intelligence model.

9. In paragraph 1, A target object classification system using a multi-class classification criterion table, characterized in that when multiple robots are used, the robot designation unit matches the target object and the robot with each other by considering the type and number of classes selected by the user and the number of the robots.

10. In paragraph 9, A target object classification system using a multi-class classification criteria table, characterized in that the robot designation unit changes the matching between the target object and the robot when the target object classification failure rate according to the matching between the target object and the robot becomes below a preset standard.

Images