Sorting system and sorting method

Abstract

To provide a sorting system and a sorting method for sorting bottles and cullet by color.SOLUTION: A sorting system includes: a separation apparatus for separating a mixture containing both bottles and cullet into the bottles and the cullet; a bottle sorting apparatus provided at a subsequent stage of the separation apparatus in order to sort the separated bottles by color; and a cullet sorting apparatus provided at a subsequent stage of the separation apparatus in order to sort the separated cullet by color.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a sorting system and method for sorting bottles and cullet by color. [Background technology]

[0002] According to the Containers and Packaging Recycling Law, municipalities that collect containers and packaging as a government service are required to separate them according to prescribed rules and hand them over to recycling businesses. Among containers and packaging, glass bottles are required to be separated by color (colorless, brown, other colors).

[0003] Conventionally, glass was separated by color (colorless, brown, other colors) manually, but handling broken glass is dangerous work and there is a labor shortage due to the decline in the working population, so automation is required. For example, Patent Document 1 proposes a system that automatically separates unbroken glass bottles. Patent Document 2 proposes a system that automatically separates broken glass bottles (hereinafter referred to as "cullet"). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-93204 A [Patent Document 2] Patent Publication No. 2021-015080 Summary of the Invention [Problem to be solved by the invention]

[0005] However, conventional automatic sorting systems for glass bottles cannot separate cullet, and conventional automatic sorting systems for cullet cannot separate bottles. Therefore, if cullet is mixed in with a collection of glass bottles, it is necessary to separate the bottles from the cullet.

[0006] The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a sorting system and a sorting method for sorting bottles and cullets by color. [Means for solving the problem]

[0007] A sorting system according to one embodiment of the present application comprises a separation device that separates a mixture containing both bottles and cullet into the bottles and the cullet, a bottle sorting device that is provided downstream of the separation device and sorts the separated bottles by color, and a cullet sorting device that is provided downstream of the separation device and sorts the separated cullet by color.

[0008] In one aspect of the present application, it is possible to separate bottles and cullet by color.

[0009] In a sorting system according to one embodiment of the present application, the bottle sorting device is equipped with a first control unit, which acquires bottle images of the separated bottles transported by a transport table, outputs bottle color information by inputting the acquired bottle images into a first learning model that has been trained to output bottle color information regarding bottle colors including colorless, brown, and other colors when a bottle image is input, and is equipped with a first sorting mechanism that sorts the bottles into colorless, brown, and other colors based on the bottle color information.

[0010] In one aspect of the present application, a bottle image is input into a first learning model, and based on the bottle color information output by the first learning model, the bottles are classified as colorless, brown, and other colors, making it possible to classify bottles with high accuracy.

[0011] In a sorting system according to one embodiment of the present application, the cullet sorting device is equipped with a second control unit, which acquires a cullet image of the separated cullet transported by a transport table, outputs cullet color information by inputting the acquired cullet image into a second learning model that has been trained to output cullet color information regarding cullet colors including colorless, brown, other colors, and labeled when a cullet image is input, and is equipped with a second sorting mechanism that sorts the cullet into colorless, brown, other colors, and labeled based on the cullet color information.

[0012] In one aspect of the present application, a cullet image is input into a second learning model, and based on the cullet color information output by the second learning model, the cullets are sorted into colorless, brown, other colors, and labeled, making it possible to sort the cullets with high accuracy.

[0013] In a separation system according to one aspect of the present application, the mixture is stored in a bag, and the system further includes a bag-breaking device that is provided upstream of the separation device and breaks the bag.

[0014] In one aspect of the present application, a bag breaking device is provided, making it possible to separate bottles and cullets while they are still contained in the bags.

[0015] A sorting system according to one embodiment of the present application includes a first collection container for collecting the colorless bottles sorted by the first sorting mechanism and the colorless cullet sorted by the second sorting mechanism, a second collection container for collecting the brown bottles sorted by the first sorting mechanism and the brown cullet sorted by the second sorting mechanism, and a third collection container for collecting the bottles of other colors sorted by the first sorting mechanism and the cullet of other colors sorted by the second sorting mechanism.

[0016] In one aspect of the present application, a first collection container is provided for collecting colorless bottles and cullet, a second collection container for collecting brown bottles and cullet, and a third collection container for collecting bottles and cullet of other colors, so that the bottles and cullet can be handed over to recycling businesses, etc.

[0017] In a sorting system according to one embodiment of the present application, the bottle color information includes a degree of certainty, and if the degree of certainty is below a threshold, the bottle image is stored and the number of images stored within a specified period is output for each color.

[0018] In one aspect of the present application, if the degree of certainty is below a threshold, the bottle image is stored and the number of images stored within a specified period is output for each color, making it possible to grasp the accuracy of the sorting.

[0019] In a sorting system according to one embodiment of the present application, the second learning model is trained using training data including colorless cullet images, brown cullet images, cullet images of other colors, and labeled cullet images with the labels still attached, as well as correct labels.

[0020] In one embodiment of the present application, cullet can be separated into colorless, brown, and other colors, and labeled cullet can also be separated.

[0021] In one aspect of the present application, the sorting system further includes a transport mechanism that transports the cullet, which has been sorted by the second sorting mechanism as being labeled, back to the cullet sorting device.

[0022] In one aspect of the present application, it is possible to reduce the proportion of cullet that could not be separated, including labeled cullet.

[0023] In a sorting system according to one embodiment of the present application, the cullets that have been sorted as labeled are re-transported into the cullet sorting device until the ratio of the cullets that have been sorted as labeled to the total number of cullets falls below a predetermined threshold value.

[0024] In one embodiment of the present application, it is possible to reduce the proportion of cullet that cannot be separated to a certain value or less.

[0025] A sorting method according to one embodiment of the present application separates a mixture containing both bottles and cullet into the bottles and the cullet using a separation device, sorts the separated bottles by color using a first sorting device, and sorts the separated cullet by color using a second sorting device.

[0026] In one aspect of the present application, it is possible to separate bottles and cullet by color. Effect of the Invention

[0027] In one aspect of the present application, it is possible to separate bottles and cullet by color. [Brief description of the drawings]

[0028] [Figure 1] FIG. 1 is an explanatory diagram showing a configuration example of a sorting system. [Diagram 2] FIG. 1 is an explanatory diagram showing an example of the configuration of an automatic raw bottle sorting device. [Diagram 3] FIG. 1 is an explanatory diagram showing a configuration example of an automatic cullet sorting device. [Figure 4] FIG. 2 is a block diagram illustrating an example of a hardware configuration of a processing device. [Diagram 5] FIG. 11 is an explanatory diagram showing an example of an image DB. [Figure 6] FIG. 13 is an explanatory diagram showing an example of a training DB. [Figure 7] FIG. 2 is an explanatory diagram showing an example of a first learning model. [Figure 8] 13 is a flowchart illustrating an example of a procedure for a first generation process. [Figure 9] 13 is a flowchart illustrating an example of a procedure for a determination process. [Figure 10] FIG. 2 is a block diagram illustrating an example of a hardware configuration of a processing device. [Figure 11] FIG. 11 is an explanatory diagram showing an example of an image DB. [Figure 12] FIG. 13 is an explanatory diagram showing an example of a training DB. [Figure 13] FIG. 11 is an explanatory diagram showing an example of a second learning model. [Figure 14] 13 is a flowchart illustrating an example of a procedure for a second generation process. [Figure 15] 13 is a flowchart illustrating an example of a procedure for a determination process. [Figure 16] 13 is a chart showing the separation results. [Figure 17] 13 is a flowchart showing an example of a procedure for a re-entry determination process. [Figure 18] FIG. 1 is an explanatory diagram showing a configuration example of an automatic cullet sorting device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] (Embodiment 1) The following describes an embodiment with reference to the drawings. FIG. 1 is an explanatory diagram showing an example of the configuration of a sorting system. The sorting system includes a bag breaking device 10, a bottle / cullet separation device 20, an automatic raw bottle sorting device 30 (bottle sorting device), an automatic cullet sorting device 40 (cullet sorting device), and a collection container 50. The sorting system is constructed in a garbage treatment facility or the like. The bag breaking device 10 is a device that breaks a garbage bag containing bottles and cullet, and sends only the bottles and cullet inside to the subsequent stage. In FIG. 1, the arrows connecting each device indicate that the bottles and cullet are transported. A conveyor, an unmanned transport device, a transport robot, or the like is used for transport. These transport mechanisms are well known technologies, so their explanations are omitted.

[0030] The bottle / cullet separator 20 separates raw bottles that have retained their shape from cullets that have broken bottles. In the bottle / cullet separator 20, guide scrapers are installed at regular intervals in the height direction on the conveyor. Raw bottles flowing on the conveyor and cullets that have retained their size (hereinafter collectively referred to as "raw bottles, etc.") are caught by the guide scrapers because they have a height equal to or greater than the interval. In contrast, cullets that are less than the interval can pass under the guide scrapers. In this way, the bottle / cullet separator 20 can separate a mixture containing both bottles and cullets into raw bottles, etc. and cullets. As a means for separating raw bottles, etc. from cullets, a form in which a punching metal, a bar screen, or a roller conveyor is installed on a vibrator may be adopted. In this case, small cullets fall through holes or gaps in the punching metal, bar screen, or roller conveyor, but raw bottles and the like do not fall to the bottom, so that raw bottles and the like can be separated from the cullets.

[0031] The automatic raw bottle sorting device 30 is a device that sorts raw bottles, etc. into colorless, brown, and other colors according to their color (bottle color). The automatic raw bottle sorting device 30 sends the sorted raw bottles, etc. to the collection container 50 according to their color.

[0032] The automatic cullet sorting device 40 is a device that sorts cullet according to its color (cullet color) into colorless, brown, and other colors. The automatic cullet sorting device 40 sends the sorted cullet to a collection container 50 according to its color.

[0033] The collection containers 50 include a colorless container 501 (first collection container), a brown container 502 (second collection container), a container of other colors 503 (third collection container), and a container with a label 504. The colorless container 501 receives colorless raw bottles, etc. from the automatic raw bottle sorting device 30, and colorless cullet from the automatic cullet sorting device 40. The brown container 502 receives brown raw bottles, etc. from the automatic raw bottle sorting device 30, and brown cullet from the automatic cullet sorting device 40. The container with a label 504 receives labeled cullet from the automatic cullet sorting device 40. The classification of raw bottles, etc. and cullet into colorless, brown, or other colors is based on the provisions of the law. The composition of colors to be classified may be changed in accordance with changes in the provisions.

[0034] FIG. 2 is an explanatory diagram showing an example of the configuration of an automatic raw bottle sorting device. The automatic raw bottle sorting device 30 (first sorting device) includes an alignment device 1, a recognition device 2 (first control unit), and a sorting device 3 (first sorting mechanism). The alignment device 1 controls the raw bottles, etc. to be sent to the downstream recognition device 2 so that they are oriented in the correct direction and spaced apart. As a method of alignment, separation by the speed difference of the conveyor can be adopted. The recognition device 2 includes a processing device 21, a camera 22, and a light 23. The image obtained by photographing the raw bottles, etc. transported by the conveyor (transport platform) with the camera 22 is input to the processing device 21. The processing device 21 identifies the color of the raw bottles, etc. The recognition device 2 transmits the identification result to the sorting device 3. The sorting device 3 includes air cylinders 31 arranged in series in the direction of travel of the conveyor and a compressor (not shown) that drives the air cylinders. Based on the identification results (colorless, brown, other colors) sent from the identification device 2, the air cylinder at the corresponding location is operated, and the raw bottles of each color are sent to a conveyor that transports the raw bottles to the collection containers 50 corresponding to colorless, brown, or other colors. As a mechanism for moving the raw bottles, other than air cylinders, an air jet device, a paddle, or a robot hand, etc. can be considered.

[0035] FIG. 3 is an explanatory diagram showing an example of the configuration of an automatic cullet sorting device. The automatic cullet sorting device 40 includes an alignment device 4 (second sorting device), and a plurality of identification devices 5 (second control unit) and a sorting device 6 (second sorting mechanism). The alignment device 4 arranges the cullets in a manner that makes them easy to identify and sort (without overlapping cullets and with spacing between them), and supplies them to the identification device 5 and sorting device 6 in the subsequent stage. As a method of alignment, a method using the speed difference between a vibration feeder and a conveyor can be adopted. The identification device 5 includes a processing device 51, a camera 52, and a light 53. The image obtained by photographing the cullet transported by the conveyor (transport platform) with the camera 52 is input to the processing device 51. The processing device 51 identifies the color of the cullet. The identification result is sent to the subsequent sorting device 6. The sorting device 6 includes a nozzle 61, an electromagnetic valve 62, and an air compressor (not shown). The nozzle 61 ejects compressed air and blows the cullet toward the conveyor that conveys the cullet to the collection container 50 at the drop port of the conveyor. The solenoid valve 62 opens and closes the nozzle 61 under the control of the identification device 5. An air compressor supplies compressed air to the nozzle 61. One identification device 5 and one sorting device 6 determine and separate each color. The first stage recognizes and separates colorless cullet. The second stage recognizes and separates brown cullet. The third stage recognizes and separates other colors. Cullets that are not recognized as colorless, brown, or other colors are collected in the labeled container 504. Most of the cullets that are not recognized as any of the above are cullets with labels attached all over. In FIG. 3, another pair of the identification device 5 and sorting device 6 may be added to separate cullets that are recognized as labeled cullet from others. In this case, the others are cullets that cannot be recognized as colored or labeled. In each sorting device 6, it is preferable to prepare a plurality of sets of nozzles 61 and solenoid valves 62 and arrange the nozzles 61 in the width direction of the conveyor. This makes it possible to sort even when a plurality of cullets flow in the width direction.

[0036] Fig. 4 is a block diagram showing an example of the hardware configuration of the processing device. The processing device 21 shown in Fig. 4 is a device provided in the identification device 2 constituting the raw bottle automatic sorting device 30. The processing device 21 includes a processing unit 211, an input / output unit 212, a storage unit 213, a communication unit 214, a display unit 215, and an operation unit 216. The processing unit 211 includes an arithmetic processing device such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit) or a GPU (Graphics Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). The processing unit 211 reads out and executes a program 21P stored in the storage unit 213 to perform various processes such as a process of controlling the camera 22 and a process of processing an image obtained by the camera 22 to identify the color of the raw bottle, which is the subject of the process.

[0037] The input / output unit 212 inputs and outputs data to and from the camera 22. The input / output unit 212 is connected to the camera 22 via, for example, a signal line, and inputs and outputs data by serial communication or parallel communication via the signal line. The input / output unit 212 transmits data such as a control command provided by the processing unit 211 to the camera 22, and provides image data input from the camera 22 to the processing unit 211. Note that communication between the camera 22 and the processing device 21 may use short-range wireless communication, for example, Bluetooth (registered trademark), an ad-hoc mode of a wireless LAN, or ZigBee (registered trademark), or may use communication using a LAN.

[0038] The storage unit 213 includes a non-volatile memory element such as a flash memory or an EEPROM (Electrically Erasable Programmable Read Only Memory). The storage unit 213 stores various programs executed by the processing unit 211 and various databases (DB) required for the processing of the processing unit 211. In this embodiment, the storage unit 213 stores the program 21P executed by the processing unit 211. The storage unit 213 also stores an image DB 2131, a training DB 2132, and a first learning model M1.

[0039] The communication unit 214 communicates with the sorting device 3. The communication unit 214 transmits to the sorting device 3 the result of identifying the color of the draft bottles performed by the processing unit 211.

[0040] The display unit 215 is configured with a liquid crystal panel, an organic EL (Electro Luminescence) display, or the like. The display unit 215 displays various images, characters, and the like based on the processing of the processing unit 211. The operation unit 216 accepts user operations and notifies the processing unit 211 of the accepted operations. For example, the operation unit 216 accepts user operations through an input device such as a mechanical button or a touch panel provided on the surface of the display unit 215. Furthermore, for example, the operation unit 216 may be an input device such as a mouse and a keyboard.

[0041] Next, the database provided in the processing device 21 will be described. FIG. 5 is an explanatory diagram showing an example of the image DB. The image DB 2131 stores images captured by the camera 22 and the recognition results of the raw bottles in the images in association with each other. The image DB 2131 includes an ID column, an image column, a shooting date and time column, a number column, a number column, a position column, a color column, and a confidence column. The ID column stores an ID for identifying an image. The image column stores images captured by the camera 22 as binary data. Image data may be stored in the storage unit 213 as a file, and a file name may be stored in the image column. It is assumed that the images to be stored are still images. When acquiring a moving image from the camera 22, the image DB 2131 does not need to store all the frames. The shooting date and time column stores the date and time when the image was captured. The time may include a frame number as well as a time code. The number column stores the number of raw bottles recognized in the image. The number column stores the sequence number of the raw bottles. The sequential numbers are numbers for the purpose of identifying each raw bottle, and the order has no special meaning. The position column stores the position of the raw bottle recognized in the image. Here, the position column stores the coordinate values ​​of the upper right corner and the lower left corner of the bounding box indicating the area in which the raw bottle appears. The center coordinates, height, and width of the bounding box may also be stored in the position column. The color column stores the color of the recognized raw bottle. The certainty column stores the certainty of the color determined by the first learning model M1. The certainty is a value greater than 0 and less than 1. A higher certainty value indicates a higher certainty.

[0042] FIG. 6 is an explanatory diagram showing an example of a training DB. The training DB 2132 stores training data used when generating or relearning the first learning model M1. The training DB 2132 includes an ID string, an image string, a number string, a number string, a position string, and a color string. The ID string stores an ID that identifies a record. The image string stores images of raw bottles as binary data. Images are stored as files in the storage unit 213, and the image string may store the name of the image file, etc. The number string stores the number of raw bottles contained in the image. The number string stores the sequence number of the raw bottles. The sequence number is used to identify each raw bottle, and the order of the numbers has no particular meaning. The position string stores the position information of a bounding box that surrounds the area of ​​the raw bottle. The color string stores the color of the raw bottle.

[0043] Next, the first learning model M1 for determining the color of the green bottle will be described. FIG. 7 is an explanatory diagram showing an example of the first learning model. FIG. 7A shows the first learning model M1 when determining the color of the green bottle. The first learning model M1 is a learning model that uses image data as input and recognizes the position and color of the green bottle contained in the image. The first learning model M1 is configured using R-CNN (Regional CNN), SSD (Single Shot MultiBox Detector), DETR (DEtection with TRansformers), YOLO (You Only Look Once), and Random Forest. In the following description, it is assumed that the first learning model M1 is configured using a neural network such as YOLO. When an image captured by the camera 22 is input to the first learning model M1, the first learning model M1 outputs the coordinates (Xmin, Ymin), (Xmax, Ymax) of the bounding box surrounding the green bottle, the confidence level, and the determined color of the green bottle (bottle color information).

[0044] FIG. 7B shows the first learning model M1 at the time of generation or re-learning. An image included in the training data is input to the first learning model M1. The first learning model M1 outputs the coordinates (Xmin, Ymin), (Xmax, Ymax) of the bounding box surrounding the raw bottle, the confidence level, and the determined color of the raw bottle. The output value of the first learning model M1 is compared with the values ​​of the position column and the color column included in the training data, and the first learning model M1 is trained so that the output value of the first learning model M1 approaches the value of the corresponding training data.

[0045] FIG. 8 is a flowchart showing an example of the procedure of the first generation process. The first generation process is a process performed when generating the first learning model M1. The processing unit 211 of the processing device 21 acquires training data from the training DB 2132 (step S1). The processing unit 211 selects one record from the training data (step S2). The processing unit 211 performs learning using one record of the selected training data (step S3). For example, the processing unit 211 inputs an image included in the training data to the first learning model M1. The first learning model M1 outputs the coordinates (Xmin, Ymin) and (Xmax, Ymax) of the bounding box surrounding the raw bottle, the confidence level, and the determined color of the raw bottle. The processing unit 211 compares these values ​​with the values ​​of the position column and the color column included in the training data, and optimizes parameters such as weights between neurons constituting the first learning model M1 so that the output value of the first learning model M1 approaches the value of the corresponding training data. The processing unit 211 determines whether there is an unprocessed record that has not been used for learning (step S4). If the processing unit 211 determines that there is an unprocessed record (YES in step S4), the processing returns to step S2 and continues. If the processing unit 211 determines that there is no unprocessed training data (NO in step S4), the processing unit 211 stores the first learning model M1 obtained by learning in the storage unit 213 (step S5) and ends the processing. Note that the first generation process (learning) may be executed in a computer other than the processing unit 21, and the result may be stored in the storage unit 213 of the processing unit 21.

[0046] FIG. 9 is a flow chart showing an example of the procedure of the judgment process. In the judgment process, the processing device 21 judges the color of the raw bottle. The processing unit 211 of the processing device 21 acquires an image from the camera 22 (step S21). The processing unit 211 inputs the acquired image to the first learning model M1 and judges the color of the raw bottle in the image (step S22). The processing unit 211 generates a control command (step S23). The processing unit 211 transmits the control command to the sorting device 3 (step S24). The control command includes, for example, the color of the raw bottle. In accordance with the control command received from the processing device 21, the sorting device 3 operates the air cylinder 31 corresponding to the color of the raw bottle, and sorts the raw bottle. The processing unit 211 associates the image used for the judgment with the judgment result and stores them in the image DB 2131 (step S25). The processing unit 211 judges whether or not to end the process (step S26). When the processing unit 211 determines not to end the process (NO in step S26), the processing returns to step S21. When the processing unit 211 determines to end the process (YES in step S26), the processing ends. The determination process is repeated without ending while the automatic raw bottle sorting device 30 is operating.

[0047] Although the color of the raw bottle is judged using the first learning model M1, the present invention is not limited to this. For example, the area of ​​the raw bottle in the image captured by the camera 22 may be recognized by pattern matching or the like, and the color may be judged from the values ​​of the R (red), G (green), and B (blue) elements of each pixel constituting the area of ​​the raw bottle. Alternatively, a light source that emits ultraviolet light or light containing ultraviolet light and a light receiving element that converts light into an electrical signal may be prepared, and the light from the light source may be irradiated onto the raw bottle, and the light receiving element may detect the reflected light or transmitted light from the raw bottle, thereby making a color judgment. Other image processing techniques for making a color judgment may be adopted.

[0048] Next, the processing device 51 provided in the identification device 5 constituting the automatic cullet sorting device 40 will be described. Fig. 10 is a block diagram showing an example of the hardware configuration of the processing device. The processing device 51 includes a processing unit 511, an input / output unit 512, a storage unit 513, a communication unit 514, a display unit 515, and an operation unit 516. The processing unit 511 includes an arithmetic processing unit such as a CPU, an MPU, or a GPU, a ROM, and a RAM. The processing unit 511 reads out and executes a program 51P stored in the storage unit 513 to perform various processes such as a process of controlling the camera 52 and a process of processing an image obtained by the camera 52 to identify the color of the cullet, which is the subject of the process.

[0049] The input / output unit 512 inputs and outputs data to and from the camera 52. The input / output unit 512 is connected to the camera 52 via, for example, a signal line, and inputs and outputs data by serial communication or parallel communication via the signal line. The input / output unit 512 transmits data such as a control command provided by the processing unit 511 to the camera 52, and provides image data input from the camera 52 to the processing unit 511. Note that communication between the camera 52 and the processing device 51 may use short-range wireless communication, for example, Bluetooth (registered trademark), an ad-hoc mode of a wireless LAN, or ZigBee (registered trademark), or may use communication using a LAN.

[0050] The storage unit 513 includes a non-volatile memory element such as a flash memory or an EEPROM. The storage unit 513 stores various programs executed by the processing unit 511 and various databases required for the processing of the processing unit 511. In this embodiment, the storage unit 513 stores the program 51P executed by the processing unit 511. The storage unit 513 also stores an image DB 5131, a training DB 5132, and a second learning model M2.

[0051] The communication unit 514 communicates with the sorting device 6. The communication unit 514 transmits the processing unit 211 to the sorting device 6 based on the identification of the color of the cullet performed by the processing unit 511.

[0052] The display unit 515 is configured with a liquid crystal panel, an organic EL display, or the like. The display unit 515 displays various images, characters, and the like based on the processing of the processing unit 511. The operation unit 516 accepts a user's operation and notifies the processing unit 511 of the accepted operation. For example, the operation unit 516 accepts a user's operation through an input device such as a mechanical button or a touch panel provided on the surface of the display unit 515. Furthermore, for example, the operation unit 516 may be an input device such as a mouse and a keyboard.

[0053] Next, the database included in the processing device 51 will be described. FIG. 11 is an explanatory diagram showing an example of an image DB. The image DB 5131 stores images captured by the camera 52 and the recognition results of the cullet in the image in association with each other. The image DB 5131 includes an ID column, an image column, a shooting date and time column, a number column, a number column, a position column, a color column, and a confidence column. The ID column stores an ID for identifying an image. The image column stores images captured by the camera 52 as binary data. Image data may be stored in the storage unit 513 as a file, and a file name may be stored in the image column. It is assumed that the images to be stored are still images. When acquiring a moving image from the camera 52, the image DB 5131 does not need to store all frames. The shooting date and time column stores the date and time when the image was captured. The time may include a frame number as well as a time code. The number column stores the number of cullets recognized in the image. The number column stores the sequential number of the cullet. The sequence number is a number for the purpose of identifying each custet, and the order has no special meaning. The position column stores the position of the custet recognized in the image. Here, the position column stores the coordinate values ​​of the upper right corner and the lower left corner of the bounding box indicating the area in which the cust is captured. The center coordinates, height, and width of the bounding box may also be stored in the position column. The color column stores the color of the recognized custet. Note that the color here includes the value "labeled" when it is determined that the entire surface of the custet is labeled. The certainty column stores the certainty of the color determined by the second learning model M2. The certainty is a value greater than 0 and less than 1. The higher the certainty value, the higher the certainty.

[0054] FIG. 12 is an explanatory diagram showing an example of a training DB. The training DB 5132 stores training data used when generating or relearning the second learning model M2. The training DB 5132 includes an ID string, an image string, a quantity string, a number string, a position string, and a color string. The ID string stores an ID for identifying a record. The image string stores an image in which a caret is included as binary data. The image may be stored as a file in the storage unit 513, and the image string may store the name of the image file or the like. The quantity string stores the number of carets included in the image. The number string stores the sequence number of the caret. The sequence number is for identifying each caret, and the sequence of the numbers has no particular meaning. The position string stores the position information of a bounding box that surrounds the area of ​​the caret. The color string stores the color (correct label) of the caret. The image used for the training data may be an image that includes one caret or multiple carets of the same color. The colorless caret image is an image in which one or multiple colorless carets are included. A brown cullet image is an image that shows one or more brown cullets. A labeled cullet image is an image that shows one or more cullets with a label affixed to one entire surface.

[0055] Next, the second learning model M2 for determining the color of a cullet will be described. FIG. 13 is an explanatory diagram showing an example of the second learning model. FIG. 13A shows the second learning model M2 when determining the color of a cullet. The second learning model M2 is a learning model that receives image data as input and recognizes the position and color of a cullet contained in the image. The second learning model M2 is configured using R-CNN, SSD, DETR, YOLO, random forest, etc. In the following description, it is assumed that the second learning model M2 is configured using a neural network such as YOLO. When an image captured by the camera 52 is input to the second learning model M2, the second learning model M2 outputs the coordinates (Xmin, Ymin), (Xmax, Ymax) of the bounding box surrounding the cullet, the confidence level, and the determined color of the cullet (cullet color information).

[0056] FIG. 13B shows the second learning model M2 when it is generated or re-learned. An image included in the training data is input to the second learning model M2. The second learning model M2 outputs the coordinates (Xmin, Ymin), (Xmax, Ymax) of the bounding box surrounding the caret, the confidence level, and the determined color of the caret. The output value of the second learning model M2 is compared with the values ​​of the position column and the color column included in the training data, and the second learning model M2 is trained so that the output value of the second learning model M2 approaches the value of the corresponding training data.

[0057] FIG. 14 is a flowchart showing an example of the procedure of the second generation process. The second generation process is a process performed when generating the second learning model M2. The processing unit 511 of the processing device 51 acquires training data from the training DB 5132 (step S41). The processing unit 511 selects one record from the training data (step S42). The processing unit 511 performs learning using one record of the selected training data (step S43). For example, the processing unit 511 inputs an image included in the training data to the second learning model M2. The second learning model M2 outputs the coordinates (Xmin, Ymin) and (Xmax, Ymax) of the bounding box surrounding the caret, the confidence level, and the determined color of the caret. The processing unit 211 compares these values ​​with the values ​​of the position column and the color column included in the training data, and optimizes parameters such as weights between neurons constituting the second learning model M2 so that the output values ​​of the second learning model M2 approach the values ​​of the corresponding training data. The processing unit 511 determines whether there is an unprocessed record that has not been used for learning (step S44). When the processing unit 511 determines that there is an unprocessed record (YES in step S44), the processing returns to step S42 and continues. When the processing unit 511 determines that there is no unprocessed training data (NO in step S44), the processing unit 511 stores the second learning model M2 obtained by learning in the memory unit 513 (step S45) and ends the processing. The generated second learning model M2 is distributed to processing units 51 other than the one that executed the second generation process. The second generation process (learning) may be executed in a computer other than the processing unit 51, and the result may be stored in the memory unit 513 of the processing unit 51.

[0058] The generated second learning model M2 classifies cullets into four types: colorless, brown, other colors, and labeled, and is therefore common to the three processing devices 51, but is not limited thereto. In the identification device 5 in charge of determining whether the cullet is colorless or not, the second learning model M2 stored in the processing device 51 may be trained to determine whether the color of the cullet is colorless or not. Similarly, a second learning model M2 for determining whether the color of the cullet is brown or not and a second learning model M2 for determining whether the color of the cullet is other colors or not may be generated and stored in the processing device 51 of the identification device 5 in charge of each color. The second learning model M2 may classify cullets into five types: colorless, brown, other colors, labeled, and indistinguishable. In the case where the second learning model M2 is generated by color, four second learning models M2 are sufficient, and cullets whose color is not a guaranteed color in all of the second learning models M2 are indistinguishable.

[0059] FIG. 15 is a flowchart showing an example of the procedure of the judgment process. In the judgment process, the processing device 51 judges the color of the cullet. The processing unit 511 of the processing device 51 acquires an image from the camera 52 (step S61). The processing unit 511 inputs the acquired image to the second learning model M2 and judges the color of the cullet in the image (step S62). The processing unit 511 judges whether the result is the color it is responsible for (step S63). If the processing unit 511 judges that the result is the color it is responsible for (YES in step S63), it generates a control command (step S64). The processing unit 511 transmits the control command to the sorting device 6 (step S65). In accordance with the control command received from the processing device 51, the sorting device 6 controls the solenoid valve 62 and releases compressed air from the nozzle 61. The released compressed air hits the cullet falling from the previous conveyor to the next conveyor for conveyor transfer, and is sent to a collection container by color. When the processing unit 511 judges that the result is not the color it is responsible for (NO in step S63), it advances the process to step S66. As a result, the cullet that is the subject of the judgment is sent to the sorting device 6 that judges other colors. Then, when the color of the cullet is not judged to be colorless, not judged to be brown, and not judged to be other colors, it is sent to the labeled container 504 as a labeled cullet. The processing unit 511 associates the image used for the judgment with the judgment result and stores them in the image DB 5131 (step S66). The processing unit 511 judges whether or not to end the process (step S67). When the processing unit 511 judges that the process is not to be ended (NO in step S67), it returns the process to step S61. When the processing unit 511 judges that the process is to be ended (YES in step S67), it ends the process. The judgment process is repeated without ending while the automatic cullet sorting device 40 is operating. As described above, when multiple sets of nozzles 61 and solenoid valves 62 are prepared and the nozzles 61 are arranged in the width direction of the conveyor, the processing unit 511 determines the solenoid valve 62 to be operated based on the coordinates of the bounding box surrounding the cullet output by the second learning model M2. Then, the processing unit 511 includes information for identifying the determined solenoid valve 62 in the control command transmitted in step S65.This allows compressed air to be discharged from the nozzle 61 according to the position of the cullet, making it possible to properly separate the cullet.

[0060] FIG. 16 is a chart showing the results of sorting. The chart in FIG. 16 shows the purity and recovery rate of brown cullet sorted by the automatic cullet sorting device 40, the purity and recovery rate of colorless cullet, and the purity and recovery rate of other colors. The purity can be calculated from the ratio of the cullet of each color collected by each conveyor (three) shown in FIG. 3 to all cullets collected by each conveyor. For example, the purity of colorless cullet can be calculated from the ratio of the colorless cullet collected at the upper stage of FIG. 3 to all cullets collected at the upper stage. From the chart in FIG. 16, the purities of brown, colorless, and other colors cullet were 90.3%, 96.9%, and 72.0%, respectively. The purities of all the above cullets were 70% or more. The recovery rate can be calculated from the ratio of the cullet of each color carried in from the sorting device 4 shown in FIG. 3 to the cullet of each color collected by each conveyor. For example, the recovery rate of colorless cullet can be calculated as the ratio of the colorless cullet recovered in the upper part of Fig. 3 to the colorless cullet brought in from the sorting device 4. From the chart in Fig. 16, the recovery rates of brown, colorless and other colored cullet were 81.5%, 72.4% and 42.5%. Of the above, the recovery rates of brown and colorless cullet were 70% or more. The sorting system can increase purity and recovery rate by repeatedly re-learning the second learning model M2, which distinguishes the color of cullet.

[0061] The sorting system, equipped with the bottle / cullet separator 20, can separate raw bottles from cullet without the need for manual labor, even if cullet is mixed in with glass bottle waste. As a result, the glass bottle sorting line can be fully automated. Automatic simultaneous sorting of raw bottles and cullet contributes to solving a problem specific to urban waste treatment facilities, that is, bottles may be unavoidably broken due to the collection system. Since cullet, which was difficult to collect manually, can be collected, the collection rate as a resource can be increased and the amount of landfilled at final disposal sites can be reduced. In the case where a label is attached to the entire surface of the cullet, the automatic cullet sorting device 40 separates it from labeled cullet instead of separating it into other colors, which makes it possible to improve the purity of the separated cullet (the proportion of the target material in the separation).

[0062] (Embodiment 2) This embodiment relates to a mode in which a cullet determined to have a label by the automatic cullet sorting device 40 is re-transported into the automatic cullet sorting device 40. This is because if the cullet placed on the conveyor for re-transportation has its unlabeled side facing up, it becomes possible to determine the color of the cullet. In the following explanation, differences from the first embodiment will be mainly described.

[0063] The sorting system of this embodiment includes a carry-in mechanism (not shown) that re-carries the labeled cullet to the automatic cullet sorting device 40. For example, the carry-in mechanism 70 includes an unmanned transport machine 71 and a vertical articulated robot 72. The unmanned transport machine 71 transports the labeled container 504 containing the labeled cullet to the vicinity of the conveyor that carries the cullet to the automatic cullet sorting device 40. The vertical articulated robot 72 picks up the labeled container 504 and places the cullet in the labeled container 504 on the conveyor that carries the cullet to the automatic cullet sorting device 40. As a result, the labeled cullet is re-carried into the automatic cullet sorting device 40, where an attempt is made to sort the labeled cullet into colorless, brown, or another color again.

[0064] Ideally, the amount of cullet determined to be labeled would be zero, but achieving this would require a certain amount of labor, which may result in low cost-effectiveness. Therefore, we will try to re-sort cullet until the ratio of labeled cullet to the total cullet falls below a certain threshold.

[0065] FIG. 17 is a flow chart showing an example of the procedure of the re-entry determination process. The re-entry determination process is executed, for example, by the processing device 51 of the identification device 5 constituting the cullet automatic sorting device 40. The processing unit 511 of the processing device 51 determines whether or not the separation of one batch has been completed (step S81). One batch refers to a collection of bottle waste. For example, it is the amount of bottle waste that has been transported by one truck. In addition, even if the bottle waste has been transported by multiple trucks, it is considered to be one batch if it has been brought in by the same local government or collection company. For example, the processing unit 511 queries a host computer that controls the entire sorting system, or the bag breaking device 10 or the bottle / cullet separation device 20, and determines whether or not the separation of one batch has been completed. When the processing unit 511 determines that the separation of one batch has been completed (YES in step S81), it calculates the ratio of labeled cullets (step S82). The ratio is calculated from the total number of the separated cullets and the number of the cullets determined to have a label, based on the judgment results stored in the image DB 5131. Alternatively, the ratio may be calculated from the total weight of the separated cullets and the weight of the cullets determined to have a label. The processing unit 511 judges whether the calculated ratio is equal to or less than a threshold (step S83). The threshold is determined in advance and stored in the storage unit 513. The threshold may be determined for each user, for example, for each municipality or each collection company that brings in bottle waste. When the processing unit 511 judges that the calculated ratio is not equal to or less than the threshold (NO in step S83), it re-carries the labeled cullets (step S84). The processing unit 511 communicates with the carry-in mechanism 70 via the communication unit 514 or via the communication unit 514 and a host computer, and operates the carry-in mechanism 70 to re-carry the labeled cullets to the automatic cullet sorting device 40. Processing unit 511 confirms that re-sorting has been performed by the automatic cullet sorting device 40 (step S85), and returns the process to step S81. Note that while the automatic cullet sorting device 40 is performing re-sorting of the cullet, the automatic raw bottle sorting device 30 may pause. Alternatively, the automatic raw bottle sorting device 30 may move on to sorting the next batch. In this case, it is necessary to ensure that the processing results, such as the judgment results, are not mixed with those of the previous and next batches. Also, in the process of combining the separated raw bottles, etc. with the cullet, it is necessary to ensure that the results of the previous and next batches are not mixed.If the processor 511 determines that the calculated ratio is equal to or less than the threshold (YES in step S83), it proceeds to sorting for the next batch (step S86) and ends the process. If the processor 511 determines that sorting of one batch is not complete (NO in step S81), it ends the process. The re-import determination process is repeatedly executed.

[0066] In this embodiment, it is possible to reduce the amount of cullet determined to be labeled. In addition, since re-sorting is performed until the ratio of cullet separated as labeled to the total separated cullet becomes equal to or less than a predetermined threshold, it is possible to realize re-sorting that is cost-effective by setting an appropriate threshold. Instead of performing re-sorting until the ratio of cullet separated as labeled becomes equal to or less than a predetermined threshold, re-sorting may be performed a predetermined number of times regardless of the ratio. The threshold may be changed for each number of re-sortings. By lowering the threshold, it is possible to prevent the time required for sorting from being unnecessarily extended by repeating the number of re-sortings.

[0067] The processing device 21 of the classification device 2 constituting the raw bottle automatic sorting device 30 stores raw bottle images that have been subjected to color judgment in the image DB 2131 in association with the judgment result, etc. The processing device 51 of the classification device 5 constituting the automatic cullet sorting device 40 stores cullet images that have been subjected to color judgment in the image DB 5131 in association with the judgment result, etc. In the above description, the conditions for raw bottle images to be stored in the image DB 2131 and the conditions for cullet images to be stored in the image DB 5131 are not specified, but conditions may be set. For example, if the certainty is equal to or lower than a threshold, the raw bottle image is stored in the image DB 2131, and the cullet image is stored in the image DB 5131. When multiple raw bottles are captured in one raw bottle image, the image is stored if at least one of the certainty levels is equal to or lower than the threshold. Similarly, when multiple cullets are captured in one cullet image, the image is stored if at least one of the certainty levels is equal to or lower than the threshold. At this time, only image regions of raw bottles or cullets whose certainty is equal to or less than a threshold value may be stored in the image DB 2131 or the image DB 5131. At this time, it is not necessary to store the number or quantity or number as attributes of the image.

[0068] Furthermore, the number of images whose confidence level falls below a threshold value during a given period may be counted for each color and output. Depending on the number of images, it becomes possible to determine the need for re-learning the first learning model M1 and the second learning model M2.

[0069] The sorting system may be provided with a management computer that manages the operating status of the bag breaking device 10, the bottle / cullet separation device 20, the automatic raw bottle sorting device 30, and the automatic cullet sorting device 40. For example, the management computer may monitor the amount of raw bottles and cullet supplied to each of the automatic raw bottle sorting device 30 and the automatic cullet sorting device 40, and if the supply amount of either is extremely low, may temporarily suspend operation of the automatic raw bottle sorting device 30 or the automatic cullet sorting device 40. The management computer may also compile the ratio of raw bottles to cullet and the amount of raw bottles and cullet discharged by color for each bottle waste generator, and report this to the generator.

[0070] (Embodiment 3) In this embodiment, an example will be described in which the sorting system re-sorts the sorted cullet. In this embodiment, the sorting system can improve the purity of the final sorted product (colorless, brown, or other color cullet) by re-sorting the colorless, brown, or other color cullet sorted by the second sorting device by the third sorting device. In the following description, differences from the first and second embodiments will be mainly described.

[0071] 18 is an explanatory diagram showing a configuration example of an automatic cullet sorting device. The automatic cullet sorting device 40 includes an upper second sorting device 5A, a middle second sorting device 5B, a lower second sorting device 5C, an upper third sorting device 6A, a middle third sorting device 6B, and a lower third sorting device 6C. The second and third sorting devices described above include a conveyor, an identification device 5, and a sorting device 6, similar to the first embodiment. Each of the third sorting devices may be appropriately removed depending on the purity of the cullet, which may be colorless, brown, or other colors.

[0072] The upper-stage second sorting device 5A separates colorless cullet from the cullet separated by the bottle / cullet separation device 20. The cullet separated by the upper-stage second sorting device 5A is transported to the upper-stage third sorting device 6A by a chute or the like. The upper-stage third sorting device 6A is provided behind the upper-stage second sorting device 5A, and separates colorless cullet again from the cullet separated by the upper-stage second sorting device 5A. The colorless cullet separated again by the upper-stage third sorting device 6A is collected in a colorless container 501. The cullet not separated again by the upper-stage third sorting device 6A is collected in a collection container (not shown).

[0073] The middle-stage second sorting device 5B separates brown cullet from the cullet that has fallen without being separated in the upper-stage second sorting device 5A. The cullet separated in the middle-stage second sorting device 5B is transported to the middle-stage third sorting device 6B by a chute or the like. The middle-stage third sorting device 6B is provided behind the middle-stage second sorting device 5B, and separates brown cullet again from the cullet separated in the middle-stage second sorting device 5B. The brown cullet separated again by the middle-stage third sorting device 6B is collected in a brown container 502. The cullet not separated again by the middle-stage third sorting device 6B is collected in a collection container not shown.

[0074] The lower-stage second sorting device 5C separates cullets of other colors from the cullets that have fallen without being separated in the middle-stage second sorting device 5B. The cullets separated in the lower-stage second sorting device 5C are transported to the lower-stage third sorting device 6C by a chute or the like. The lower-stage third sorting device 6C is provided behind the lower-stage second sorting device 5C, and separates cullets of other colors again from the cullets separated in the lower-stage second sorting device 5C. The cullets of other colors again separated by the lower-stage third sorting device 6C are collected in the other color container 503. The cullets that have not been separated again by the lower-stage third sorting device 6C are collected in a collection container (not shown). The cullets that have fallen without being separated in the lower-stage second sorting device 5C are collected in the labeled container 504. The cullets collected in the collection container (not shown) may be transported again to the alignment device 4 by a conveying device or the like.

[0075] The sorting system can further sort the sorted cullet to improve the purity of the final sort.

[0076] The technical features (constituent elements) described in each embodiment can be combined with each other, and by combining them, new technical features can be formed. The embodiments disclosed herein are illustrative in all respects and should not be considered as limiting. The scope of the present invention is indicated by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims. In addition, the claims use a format in which a claim cites two or more other claims (multiple claim format), but are not limited to this. A format in which a multiple claim cites at least one multiple claim (multiple multiple claim) may also be used. [Explanation of symbols]

[0077] 10: Bag breaking device 20: Cullet separator 30: Automatic bottle sorting device (first sorting device / bottle sorting device) 1: Alignment device 2: Identification device (first control unit) 21: Processing equipment 211: Processing section 212: Input / output section 213: Storage section 2131: Image DB 2132 :Training DB M1: First learning model 214: Communications Department 215:Display section 216:Operation unit Page 21: Program 22: Camera 23: Lighting 3: Sorting device (first sorting mechanism) 31: Air cylinder 40: Cullet automatic sorting device (second sorting device / cullet sorting device) 4: Alignment device 5: Identification device (second control unit) 51: Processing equipment 511: Processing section 512: Input / output section 513: Storage section 5131: Image DB 5132 :Training DB M2: Second learning model 514: Communications Department 515:Display section 516:Operation unit Page 51: Program 52: Camera 53: Lighting 6: Sorting device (second sorting mechanism) 61: Nozzle 62: Solenoid valve 50: Collection container 501 :Colorless container 502: Brown container 503: Other color containers 504: Labeled containers 5A: Upper 2nd sorting device 5B: Middle stage 2nd sorting device 5C: Lower 2nd sorting device 6A: Upper 3rd sorting device 6B: Middle stage 3rd sorting device 6C: Lower 3rd sorting device

Claims

1. a separation device for separating a mixture containing both bottles and cullet into said bottles and said cullet; a bottle sorting device provided downstream of the separating device for sorting the separated bottles by color; a cullet sorting device that is provided downstream of the separating device and sorts the separated cullet by color; A separation system comprising:

2. The bottle sorting device includes a first control unit, The first control unit is acquiring bottle images of the separated bottles being transported by a transport platform; The acquired bottle image is input to a first learning model that has been trained to output bottle color information regarding bottle colors including colorless, brown, and other colors when a bottle image is input, thereby outputting bottle color information; A first sorting mechanism is provided for sorting the bottles into colorless, brown, and other colors based on the bottle color information. The sorting system of claim 1 .

3. The cullet sorting device includes a second control unit, The second control unit is acquiring a cullet image of the separated cullet being transported by a transport platform; A second learning model is trained to output cullet color information regarding cullet colors including colorless, brown, other colors, and labeled colors when a cullet image is input, by inputting the acquired cullet image to the second learning model; and A second sorting mechanism is provided for sorting the cullet into colorless, brown, other colors, and labeled, based on the cullet color information. The sorting system according to claim 1 or 2.

4. The mixture is contained in a bag, A bag breaking device that is provided in front of the separating device and breaks the bags. Further comprising The sorting system according to claim 1 or 2.

5. a first collection container for collecting the colorless bottles separated by the first sorting mechanism and the colorless cullet separated by the second sorting mechanism; a second collection container for collecting the brown bottles separated by the first sorting mechanism and the brown cullet separated by the second sorting mechanism; a third collection container for collecting the bottles of other colors separated by the first sorting mechanism and the cullet of other colors separated by the second sorting mechanism. The separation system according to claim 3 which recites claim 2.

6. The bottle color information includes a confidence level, and if the confidence level is equal to or lower than a threshold level, the bottle image is stored; Output the number of images stored for each color within a specified period The sorting system of claim 2.

7. The second learning model is trained using training data including colorless cullet images, brown cullet images, cullet images of other colors, and labeled cullet images with the labels still attached, as well as correct labels. The sorting system according to claim 3.

8. The cullet sorting device further includes a carrying mechanism that carries the cullet sorted by the second sorting mechanism as being labeled back into the cullet sorting device. The sorting system according to claim 3.

9. The cullet sorting device comprises: an upper stage second sorting device for sorting colorless cullet from the separated cullet; a middle-stage second sorting device for sorting brown cullet from cullet that has fallen without being sorted in the upper-stage second sorting device; a lower-stage second sorting device for sorting cullets of other colors from the cullets that have fallen without being sorted in the middle-stage second sorting device; an upper-stage third sorting device provided downstream of the upper-stage second sorting device for re-sorting colorless cullet from the cullet sorted by the upper-stage second sorting device; a middle-stage third sorting device provided downstream of the middle-stage second sorting device for re-sorting brown cullet from the cullet sorted by the middle-stage second sorting device; The separation system of claim 1 .

10. The cullets classified as labeled are re-transported into the cullet classification device until a ratio of the cullets classified as labeled to the total number of cullets becomes equal to or less than a predetermined threshold value. The sorting system of claim 8.

11. a separator for separating the mixture containing both bottles and cullet into the bottles and the cullet; A first sorting device sorts the separated bottles by color; A second sorting device separates the separated cullet by color. Separation method.

Images