Control device

Abstract

A container sorting system 1 has: a camera 20 for capturing an image of a group of waste materials on a belt conveyor 40 on which the group of waste materials are conveyed; an image recognition unit 12 for recognizing the type and existence region of a container included within the group of waste materials on the basis of the image captured by the camera 20; an area centroid calculation unit 13 for calculating first coordinates that are the coordinates of the area centroid of the container in the image captured by the camera 20; an extraction point coordinates calculation unit 14 for calculating second coordinates that are the coordinates moved in the bottom direction of the container from the first coordinates by a prescribed amount; and an extraction unit 31 for using the second coordinates as the extraction point for container extraction to extract the container from among the group of waste materials that are conveyed on the belt conveyor 40.

Description

Container sorting system

[0001] The present disclosure relates to a container sorting system.

[0002] At waste disposal sites, large amounts of waste are processed daily on conveyor belts. At the waste disposal sites, waste is sorted by hand. While sorting waste is a simple task, it places a heavy burden on the workers who sort the waste (hereinafter referred to as "sorters"). Therefore, devices that automatically sort waste (hereinafter referred to as "waste sorting devices") have been developed.

[0003] Patent No. 6854995 JP 2012-158406 A

[0004] When the work that was previously performed by a sorting worker is performed by a waste sorting device instead of the sorting worker, the waste sorting device may recognize each piece of waste flowing on the conveyor belt, and based on the recognition results, use a robotic hand or suction pad to extract desired waste (hereinafter sometimes referred to as "desired waste") from the group of waste flowing on the conveyor belt.

[0005] Here, if the desired waste is a bottle, when the waste sorting device extracts the bottle from the waste mass, if the position where the robot hand grips the bottle (hereinafter referred to as the "gripping position") or the position where the suction pad sucks the bottle (hereinafter referred to as the "suction position") is shifted from the center of gravity of the bottle's weight, the bottle will tilt after extraction, and there is a risk that the bottle will fall from the robot hand or suction pad while being transported by the robot hand or suction pad.

[0006] Therefore, the present disclosure proposes a technology that allows the container to be transported without tilting after extraction.

[0007] The container sorting system of the present disclosure includes a camera, a recognition unit, a first calculation unit, a second calculation unit, and an extraction unit. The camera captures an image of a group of waste materials on a transport path along which the group of waste materials is transported. The recognition unit recognizes the location and type of containers contained in the group of waste materials based on the image. The first calculation unit calculates a first coordinate, which is the coordinate of the area center of gravity of the container in the image. The second calculation unit calculates a second coordinate, which is a coordinate shifted from the first coordinate by a specified amount toward the bottom of the container. The extraction unit uses the second coordinate as an extraction point when extracting the container, and extracts the container from the group of waste materials transported on the transport path.

[0008] According to the disclosed technology, the container can be transported without tilting after extraction.

[0009] FIG. 1 is a diagram illustrating an example configuration of a container sorting system according to a first embodiment of the present disclosure. FIG. 2 is a diagram illustrating an example configuration of a control device and a sorting device according to a first embodiment of the present disclosure. FIG. 3 is a diagram illustrating an example of an extraction unit according to a first embodiment of the present disclosure. FIG. 4 is a diagram illustrating an example of an extraction unit according to a first embodiment of the present disclosure. FIG. 5 is a diagram illustrating an example of a processing procedure in the container sorting system according to a first embodiment of the present disclosure. FIG. 6 is a diagram illustrating an example operation of the container sorting system according to a first embodiment of the present disclosure. FIG. 7 is a diagram illustrating an example of a specified amount determination table according to a first embodiment of the present disclosure.

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the following embodiments, the same components are designated by the same reference numerals.

[0011] First Embodiment <Configuration of Container Sorting System> FIG. 1 is a diagram illustrating an example of the configuration of a container sorting system according to a first embodiment of the present disclosure.

[0012] 1, the container sorting system 1 includes a control device 10, a camera 20, a sorting device 30, and a belt conveyor 40. The control device 10, the camera 20, and the sorting device 30 are connected to each other via a network.

[0013] The following description will be given, as an example, of a case where the container sorting system 1 shown in FIG. 1 is installed at a waste disposal site where waste flows on a belt conveyor 40. That is, the following description will be given, as an example, of a case where the objects to be sorted by the container sorting system 1 are waste. Also, the following description will be given, as an example, of a case where the desired waste is bottles. Bottles are an example of a container, and the disclosed technology can be applied to containers other than bottles. That is, the objects to be sorted by the container sorting system 1 are not limited to bottles, and the container sorting system 1 can be used for various containers.

[0014] The belt conveyor 40 transports the waste mass placed on the belt conveyor 40 in a transport direction CD (+X direction). In other words, the belt conveyor 40 forms a transport path along which the waste mass is transported in the transport direction CD.

[0015] The camera 20 is disposed above the belt conveyor 40 along which the waste mass is transported, has a predetermined angle of view, and captures an image of a predetermined area on the upper surface of the belt conveyor 40 from above the belt conveyor 40 at a constant frame rate. Therefore, the image captured by the camera 20 becomes an image of the waste mass (hereinafter sometimes referred to as a "waste mass image"). The waste mass image is transmitted from the camera 20 to the control device 10.

[0016] The control device 10 controls the operation of the sorting device 30 based on the waste mass image.

[0017] Under the control of the control device 10, the sorting device 30 sorts bottles by extracting them from a group of waste being transported in the transport direction CD by the belt conveyor 40, and transports the extracted bottles out of the belt conveyor 40.

[0018] <Configuration of Control Device and Sorting Device> Figure 2 is a diagram showing an example configuration of a control device and a sorting device according to a first embodiment of the present disclosure. In Figure 2, the control device 10 has an image processing unit 11, a learned model storage unit 15, and a sorting device control unit 16. The image processing unit 11 has an image recognition unit 12, an area centroid calculation unit 13, and an extraction point coordinate calculation unit 14. The sorting device 30 has an extraction unit 31. A waste group image captured by the camera 20 is input to the image processing unit 11.

[0019] 3 and 4 are diagrams illustrating an example of an extraction unit according to the first embodiment of the present disclosure. For example, the extraction unit 31 may be formed using a robot hand 311 as shown in FIG. 3 . Alternatively, for example, the extraction unit 31 may be formed using a suction pad 312 as shown in FIG. 4 . The sorting device 30 uses the extraction unit 31 to extract bottles from the waste material transported on the belt conveyor 40 under control of the sorting device control unit 16. The extraction unit 31 is movable in the ±Y and ±Z directions, and transports the bottle extracted from the waste material along the side of the belt conveyor 40 to a predetermined box located outside the belt conveyor 40 by lifting the bottle in the +Z direction and moving it in the +Y or −Y direction.

[0020] <Processing of Container Sorting System> FIG. 5 is a diagram illustrating an example of a processing procedure in the container sorting system according to the first embodiment of the present disclosure.

[0021] In FIG. 5, in step S100, the camera 20 takes an image of the waste mass.

[0022] Next, in step S105, the image recognition unit 12 recognizes the area and type of bottles included in the waste group by image recognition based on the waste group image. Hereinafter, the area of ​​bottles recognized by the image recognition unit 12 will be referred to as the "bottle area," and the type of bottle recognized by the image recognition unit 12 will be referred to as the "bottle type." The image recognition unit 12 recognizes the bottle area and bottle type using a trained model stored in the trained model storage unit 15. The image recognition unit 12 assigns the bottle area and bottle type to images of bottles present in the waste group image (hereinafter sometimes referred to as "bottle images") as information indicating the characteristics of the bottles (hereinafter sometimes referred to as "characteristic information").

[0023] Next, in step S110, the area centroid calculation unit 13 calculates the coordinates of the area centroid of the bottle (hereinafter, sometimes referred to as "area centroid coordinates"). The area centroid calculation unit 13 calculates the coordinates of the center of gravity of the bottle region recognized by the image recognition unit 12 as the area centroid coordinates.

[0024] Next, in step S115, the extraction point coordinate calculation unit 14 determines whether the bottle type recognized by the image recognition unit 12 is a PET bottle. If the bottle type is a PET bottle (step S115: Yes), the process proceeds to step S130, and if the bottle type is not a PET bottle (step S115: No), the process proceeds to step S120.

[0025] Next, in step S120, the extraction point coordinate calculation unit 14 calculates the total length of the bottle in the longitudinal direction (hereinafter sometimes referred to as the "bottle length") based on the bottle area recognized by the image recognition unit 12.

[0026] Next, in step S125, the extraction point coordinate calculation unit 14 determines whether the bottle length is less than the threshold value TH1. If the bottle length is less than the threshold value TH1 (step S125: Yes), the process proceeds to step S130. If the bottle length is equal to or greater than the threshold value TH1 (step S125: No), the process proceeds to step S135.

[0027] Next, in step S130, the extraction point coordinate calculation unit 14 sets the area centroid coordinates calculated in step S110 as coordinates indicating the extraction point when the extraction unit 31 extracts the bottle (hereinafter sometimes referred to as "extraction point coordinates"), and outputs the set extraction point coordinates to the sorting device control unit 16.

[0028] On the other hand, in step S135, the extraction point coordinate calculation unit 14 calculates coordinates indicating the center of gravity of the bottle weight (hereinafter, sometimes referred to as "weight center of gravity coordinates"). The extraction point coordinate calculation unit 14 calculates the weight center of gravity coordinates by moving the area center of gravity coordinates calculated by the area center of gravity calculation unit 13 by a specified amount toward the bottom of the bottle.

[0029] Next, in step S140 , the extraction point coordinate calculation unit 14 sets the weight center of gravity coordinate calculated in step S110 as the extraction point coordinate, and outputs the set extraction point coordinate to the sorting device control unit 16 .

[0030] Since the processes of steps S105 to S140 are all performed based on the waste group image, the extraction point coordinates set by the process of step S130 or step S140 are coordinates in the coordinate system of the waste group image, that is, the coordinate system of the camera 20 (hereinafter sometimes referred to as the "camera coordinate system"). Therefore, in step S145, the sorting device control unit 16 converts the extraction point coordinates in the camera coordinate system set by the process of step S130 or step S140 into extraction point coordinates in the coordinate system of the sorting device 30. The sorting device control unit 16 outputs a control signal including the converted extraction point coordinates to the sorting device 30.

[0031] Then, in step S150, the sorting device 30 moves the extraction unit 31 directly above the extraction point coordinates indicated in the control signal, and the extraction unit 31 uses the extraction point coordinates as the extraction point to extract a bottle from the waste material being transported on the belt conveyor 40. If the extraction unit 31 is formed using a robot hand 311, the extraction unit 31 sets the extraction point on the bottle to the gripping position of the robot hand 311 relative to the bottle and extracts the bottle by gripping with the robot hand 311. If the extraction unit 31 is formed using a suction pad 312, the extraction unit 31 sets the extraction point on the bottle to the suction position of the suction pad 312 relative to the bottle and extracts the bottle by suction with the suction pad 312.

[0032] <Operation of Container Sorting System> FIG. 6 is a diagram showing an example of operation of the container sorting system according to the first embodiment of the present disclosure.

[0033] As shown in FIG. 6 , the image recognition unit 12 assigns feature information to the bottle image BI, including label information LA indicating the bottle type and outline information CO indicating the bottle region (step S105). Bottle types are, for example, classified into champagne bottles, wine bottles, PET bottles, and other bottles. Furthermore, with the long side of the rectangular waste group image as the X axis and the short side as the Y axis, the outline information CO is formed by multiple coordinate points (x0, y0), (x1, y1), ..., (xn, yn). In other words, the area surrounded by lines connecting the multiple coordinate points (x0, y0), (x1, y1), ..., (xn, yn) that form the outline information CO is the bottle region.

[0034] The area center of gravity calculation unit 13 calculates the area center of gravity coordinates AC(Xa, Ya) of the bottle region based on the contour information CO (step S110).

[0035] The extraction point coordinate calculation unit 14 calculates the bottle length L, which is the total length of the bottle region in the longitudinal direction, based on the contour information CO (step S120).

[0036] The extraction point coordinate calculation unit 14 determines the bottom BP of the bottle based on the bottle image BI, and calculates the weight center of gravity coordinate WC(Xw, Yw) by moving the area center of gravity coordinate AC(Xa, Ya) toward the bottom BP of the bottle by a specified amount Δl (step S135). The extraction point coordinate calculation unit 14 determines the bottom BP of the bottle, for example, using a trained model stored in the trained model storage unit 15.

[0037] In the case of a PET bottle, the difference between the thickness of the bottom and the thickness of the rest of the bottle is small, so the center of gravity of the weight is almost the same as the center of gravity of the area. Therefore, when the bottle type is a PET bottle, the extraction point coordinate calculation unit 14 sets the coordinates of the center of gravity of the area as the extraction point coordinates (step S115: Yes, step S130). In other words, when the bottle type is a PET bottle, the extraction point coordinate calculation unit 14 sets the specified amount Δl to 0.

[0038] Furthermore, when the bottle length L is short, the weight center of gravity is approximately the same as the area center of gravity. Therefore, when the bottle length L is less than the threshold value TH1, the extraction point coordinate calculation unit 14 sets the area center of gravity coordinate as the extraction point coordinate (step S125: Yes, step S130). In other words, when the bottle length L is less than the threshold value TH1, the extraction point coordinate calculation unit 14 sets the specified amount Δl to 0.

[0039] Furthermore, when calculating the weight center of gravity coordinates (step S135), the extraction point coordinate calculation unit 14 controls the specified amount Δl based on the bottle type. FIG. 7 is a diagram showing an example of a specified amount determination table according to the first embodiment of the present disclosure. As shown in FIG. 7, the specified amount determination table TA associates bottle types with specified amounts Δl. The extraction point coordinate calculation unit 14 has the specified amount determination table TA and controls the specified amount Δl in accordance with the specified amount determination table TA based on the bottle type. In the specified amount determination table TA, the specified amount Δl is set as a percentage [%] of the bottle length L.

[0040] Generally, the thickness of the bottom of a champagne bottle is greater than that of a wine bottle. The thickness of the bottom of a champagne bottle and the thickness of the bottom of a wine bottle are also greater than those of other bottles. In other words, the difference between the area center of gravity and the weight center of gravity of a champagne bottle is greater than that of a wine bottle. The difference between the area center of gravity and the weight center of gravity of a wine bottle is also greater than that of other bottles. Therefore, when the bottle type is a champagne bottle, the extraction point coordinate calculation unit 14 sets the specified amount Δl to 6% of the bottle length L in accordance with the specified amount determination table TA. When the bottle type is a wine bottle, the extraction point coordinate calculation unit 14 sets the specified amount Δl to 4% of the bottle length L in accordance with the specified amount determination table TA. When the bottle type is other bottles, the extraction point coordinate calculation unit 14 sets the specified amount Δl to 2% of the bottle length L in accordance with the specified amount determination table TA.

[0041] In this way, the extraction point coordinate calculation unit 14 controls the specified amount Δl based on the bottle type and bottle length L.

[0042] The first embodiment has been described above.

[0043] [Example 2] The trained model storage unit 15 is realized as hardware, for example, by a memory or a storage. The image processing unit 11 and the sorting device control unit 16 are realized as hardware, for example, by a processor such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or an ASIC (Application Specific Integrated Circuit).

[0044] Furthermore, all or part of the processes described above in the image processing unit 11 and the sorting device control unit 16 may be realized by having a processor execute a program corresponding to each process. For example, a program corresponding to each process described above may be stored in a memory included in the control device 10, and the program may be read from the memory and executed by the processor. Alternatively, the program may be stored in a program server connected to the control device 10 via an arbitrary network, downloaded from the program server to the control device 10, and executed therein, or stored in a recording medium readable by the control device 10, read from the recording medium, and executed therein. Examples of recording media readable by the control device 10 include portable storage media such as memory cards, USB memories, SD cards, flexible disks, magneto-optical disks, CD-ROMs, and DVDs.

[0045] The second embodiment has been described above.

[0046] As described above, the container sorting system (container sorting system 1 of the embodiment) of the present disclosure includes a camera (camera 20 of the embodiment), a recognition unit (image recognition unit 12 of the embodiment), a first calculation unit (area centroid calculation unit 13 of the embodiment), a second calculation unit (extraction point coordinate calculation unit 14 of the embodiment), and an extraction unit (extraction unit 31 of the embodiment). The camera captures an image of the waste mass on a conveyance path (belt conveyor 40 of the embodiment) along which the waste mass is transported. The recognition unit recognizes the presence area and type of containers included in the waste mass based on the image captured by the camera. The first calculation unit calculates a first coordinate (area centroid coordinate of the embodiment), which is the coordinate of the area centroid of the container in the image captured by the camera. The second calculation unit calculates a second coordinate (weight centroid coordinate of the embodiment), which is the coordinate shifted a specified amount from the first coordinate toward the bottom of the container. The extraction unit uses the second coordinate as an extraction point when extracting the container, and extracts the container from the waste mass transported along the conveyance path.

[0047] For example, the recognition unit uses a trained model to recognize the area and type of container.

[0048] In this way, the extraction point of the container can be set at the center of gravity of the container's weight, allowing the extracted container to be transported without tilting. Furthermore, since the second coordinate is calculated based on an image captured by a camera, the second coordinate can be calculated without using a measuring device or sensor to measure the weight of the container. Furthermore, since the extracted container can be transported without tilting, the height to which the extracted container is lifted in the +Z direction can be made smaller than when the container is tilted.

[0049] The second calculation unit also controls the specified amount based on the type of container.

[0050] This allows the container to be transported without tilting after extraction, regardless of the type of container.

[0051] The second calculation unit also controls the specified amount based on the length of the container in the longitudinal direction.

[0052] This allows the container to be transported without tilting after extraction, regardless of the longitudinal length of the container.

[0053] REFERENCE SIGNS LIST 1 Container sorting system 10 Control device 11 Image processing unit 12 Image recognition unit 13 Area center of gravity calculation unit 14 Extraction point coordinate calculation unit 15 Trained model storage unit 16 Sorting device control unit 20 Camera 30 Sorting device 31 Extraction unit

Claims

1. A first calculation unit that calculates a first coordinate, which is the coordinate of the center of area gravity of the container in an image taken by a camera that takes an image of a waste group including a container on a transport path along which the waste group is transported; a second calculation unit that calculates a second coordinate, which is a coordinate shifted a specified amount from the first coordinate toward the bottom of the container, as the coordinate of an extraction point when the container is extracted from the waste material transported along the transport path; A control device comprising:

2. A recognition unit that recognizes an area and a type of container contained in the waste group based on the image. The control device of claim 1 further comprising:

3. The recognition unit recognizes the existence region and the type by using a trained model. The control device according to claim 2.

4. The second calculation unit controls the specified amount based on the type. The control device according to claim 2.

5. The second calculation unit controls the specified amount based on the length of the container in the longitudinal direction. The control device according to claim 1 .