control device

Abstract

A container sorting system 1 has: a camera 20 for capturing an image of a group of waste materials on a belt conveyor 40 on which the group of waste materials are conveyed; an image recognition unit 12 for recognizing the type and existence region of a container included within the group of waste materials on the basis of the image captured by the camera 20; an area centroid calculation unit 13 for calculating first coordinates that are the coordinates of the area centroid of the container in the image captured by the camera 20; an extraction point coordinates calculation unit 14 for calculating second coordinates that are the coordinates moved in the bottom direction of the container from the first coordinates by a prescribed amount; and an extraction unit 31 for using the second coordinates as the extraction point for container extraction to extract the container from among the group of waste materials that are conveyed on the belt conveyor 40.

Description

Technical Field

[0001] This disclosure relates to control device .

Background Art

[0002] At waste treatment plants, a large amount of waste flows on a belt conveyor every day and is being processed. At the site where the waste is processed, the waste is sorted by hand. Although the waste sorting work is a simple task, the burden on the workers who sort the waste (hereinafter sometimes referred to as "sorting workers") is large, so a device that automatically sorts waste (hereinafter sometimes referred to as a "waste sorting device") has been developed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] When the waste sorting device is made to perform the work that the sorting worker has been doing instead of the sorting worker, the waste sorting device recognizes each piece of waste flowing on the belt conveyor and, based on the recognition result, uses a robot hand or a suction pad to extract a desired piece of waste (hereinafter sometimes referred to as "desired waste") from the group of waste flowing on the belt conveyor.

[0005] In this case, if the desired waste is a bottle, and the waste sorting device extracts the bottle from the waste group, if the position where the robot hand grips the bottle (hereinafter sometimes referred to as the "gripping position") or the position where the suction pad sucks the bottle (hereinafter sometimes referred to as the "suction position") is off from the center of gravity of the bottle, the extracted bottle may tilt, and there is a risk that the bottle being transported by the robot hand or suction pad may fall from the robot hand or suction pad.

[0006] Therefore, this disclosure proposes a technology that allows the container to be transported without tilting after extraction. [Means for solving the problem]

[0007] The container sorting system of this disclosure comprises a camera, a recognition unit, a first calculation unit, a second calculation unit, and an extraction unit. The camera captures an image of the waste group on a transport path through which the waste group is being transported. The recognition unit recognizes the area and type of containers included in the waste group based on the image. The first calculation unit calculates a first coordinate, which is the coordinate of the centroid of the area of ​​the container in the image. The second calculation unit calculates a second coordinate, which is the coordinate obtained by moving a specified amount from the first coordinate toward the bottom of the container. The extraction unit uses the second coordinate as the extraction point for the container and extracts the container from the waste group being transported on the transport path. [Effects of the Invention]

[0008] According to the disclosed technology, the container can be transported without tilting after extraction. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 shows an example of the configuration of the container sorting system of Embodiment 1 of this disclosure. [Figure 2] Figure 2 shows an example of the configuration of the control device and sorting device of Embodiment 1 of the present disclosure. [Figure 3] Figure 3 shows an example of the extraction unit in Embodiment 1 of this disclosure. [Figure 4] Figure 4 shows an example of the extraction unit in Embodiment 1 of this disclosure. [Figure 5] Figure 5 shows an example of the processing procedure in the container sorting system of Embodiment 1 of this disclosure. [Figure 6] Figure 6 shows an example of the operation of the container sorting system of Embodiment 1 of this disclosure. [Figure 7] Figure 7 shows an example of a specified amount determination table for Example 1 of this disclosure. [Modes for carrying out the invention]

[0010] The embodiments of this disclosure will be described below with reference to the drawings. In the following embodiments, identical components will be denoted by the same reference numerals.

[0011] [Example 1] <Configuration of the container sorting system> Figure 1 shows an example of the configuration of the container sorting system of Embodiment 1 of this disclosure.

[0012] In Figure 1, the container sorting system 1 includes a control device 10, a camera 20, a sorting device 30, and a belt conveyor 40. The control device 10, the camera 20, and the sorting device 30 are connected to each other via a network.

[0013] In the following explanation, we will use the case where the container sorting system 1 shown in Figure 1 is installed in a waste treatment plant where a group of waste materials flows on a belt conveyor 40 as an example. In other words, in the following explanation, we will use the case where the object to be sorted by the container sorting system 1 is waste as an example. Furthermore, in the following explanation, we will use the case where the desired waste material is bottles as an example. Bottles are just one example of a container, and the disclosed technology is applicable to containers other than bottles. In other words, the object to be sorted by the container sorting system 1 is not limited to bottles, and the container sorting system 1 can be used for various containers.

[0014] The belt conveyor 40 conveys a group of wastes placed thereon in the conveying direction CD (+X direction). That is, the belt conveyor 40 forms a conveying path along which the group of wastes is conveyed in the conveying direction CD.

[0015] The camera 20 is disposed above the belt conveyor 40 on which the group of wastes is conveyed, has a predetermined imaging angle, and captures a predetermined area on the upper surface of the belt conveyor 40 from above the belt conveyor 40 at a constant frame rate. Thus, the image captured by the camera 20 becomes an image of the group of wastes (hereinafter sometimes referred to as "group-of-wastes image"). The group-of-wastes image is transmitted from the camera 20 to the control device 10.

[0016] The control device 10 controls the operation of the sorting device 30 based on the group-of-wastes image.

[0017] The sorting device 30 sorts by extracting bottles from the group of wastes conveyed in the conveying direction CD by the belt conveyor 40 under the control of the control device 10, and conveys the extracted bottles outside the belt conveyor 40.

[0018] <Configuration of the control device and the sorting device> FIG. 2 is a diagram showing a configuration example of the control device and the sorting device according to Embodiment 1 of the present disclosure. In FIG. 2, the control device 10 includes an image processing unit 11, a learned model storage unit 15, and a sorting device control unit 16. The image processing unit 11 includes an image recognition unit 12, an area centroid calculation unit 13, and an extraction point coordinate calculation unit 14. The sorting device 30 includes an extraction unit 31. The group-of-wastes image captured by the camera 20 is input to the image processing unit 11.

[0019] Figures 3 and 4 show an example of the extraction unit of Embodiment 1 of the present disclosure. For example, the extraction unit 31 is formed using a robot hand 311, as shown in Figure 3. Alternatively, the extraction unit 31 is formed using a suction pad 312, as shown in Figure 4. The sorting device 30 sorts by using the extraction unit 31 to extract bottles from a group of waste being transported on the belt conveyor 40, according to control from the sorting device control unit 16. The extraction unit 31 is movable in the ±Y and ±Z directions, and transports the bottles extracted from the group of waste by lifting them in the +Z direction and moving them in the +Y or -Y direction, along the side of the belt conveyor 40 to a predetermined box located outside the belt conveyor 40.

[0020] <Processing in the container sorting system> Figure 5 shows an example of the processing procedure in the container sorting system of Embodiment 1 of this disclosure.

[0021] In Figure 5, in step S100, camera 20 captures an image of the waste group.

[0022] Next, in step S105, the image recognition unit 12 recognizes the location and type of bottles included in the waste group by image recognition based on the waste group image. Hereinafter, the location of bottles recognized by the image recognition unit 12 may be referred to as the "bottle region," and the type of bottle recognized by the image recognition unit 12 may be referred to as the "bottle type." The image recognition unit 12 recognizes the bottle region and bottle type using the trained model stored in the trained model storage unit 15. The image recognition unit 12 assigns the bottle region and bottle type to the images of bottles present in the waste group image (hereinafter sometimes referred to as "bottle images") as information indicating the characteristics of the bottles (hereinafter sometimes referred to as "feature information").

[0023] Next, in step S110, the area centroid calculation unit 13 calculates the coordinates of the area centroid of the bottle (hereinafter sometimes referred to as "area centroid coordinates"). The area centroid calculation unit 13 calculates the coordinates of the centroid of the bottle region recognized by the image recognition unit 12 as the area centroid coordinates.

[0024] Next, in step S115, the extraction point coordinate calculation unit 14 determines whether the bottle type recognized by the image recognition unit 12 is a PET bottle or not. If the bottle type is a PET bottle (step S115: Yes), the process proceeds to step S130; if the bottle type is not a PET bottle (step S115: No), the process proceeds to step S120.

[0025] Next, in step S120, the extraction point coordinate calculation unit 14 calculates the total length of the bottle in the longitudinal direction (hereinafter sometimes referred to as "bottle length") based on the bottle region recognized by the image recognition unit 12.

[0026] Next, in step S125, the extraction point coordinate calculation unit 14 determines whether the bottle length is less than the threshold TH1. If the bottle length is less than the threshold TH1 (step S125: Yes), the process proceeds to step S130; if the bottle length is greater than or equal to the threshold TH1 (step S125: No), the process proceeds to step S135.

[0027] Next, in step S130, the extraction point coordinate calculation unit 14 sets the area centroid coordinate calculated in step S110 as the coordinate indicating the extraction point when the bottle is extracted by the extraction unit 31 (hereinafter sometimes referred to as the "extraction point coordinate"), and outputs the set extraction point coordinate to the sorting device control unit 16.

[0028] On the other hand, in step S135, the extraction point coordinate calculation unit 14 calculates the coordinates indicating the center of gravity of the bottle's weight (hereinafter sometimes referred to as the "weight center of gravity coordinates"). The extraction point coordinate calculation unit 14 calculates the weight center of gravity coordinates by moving the area center of gravity coordinates calculated by the area center of gravity calculation unit 13 by a specified amount toward the bottom of the bottle.

[0029] Next, in step S140, the extraction point coordinate calculation unit 14 sets the center of gravity coordinate calculated in step S110 as the extraction point coordinate and outputs the set extraction point coordinate to the sorting device control unit 16.

[0030] Since the processing in steps S105 to S140 is all based on the waste group image, the extracted point coordinates set by the processing in step S130 or step S140 are coordinates in the coordinate system of the waste group image, that is, the coordinate system of camera 20 (hereinafter sometimes referred to as the "camera coordinate system"). Therefore, in step S145, the sorting device control unit 16 converts the extracted point coordinates in the camera coordinate system set by the processing in step S130 or step S140 to the extracted point coordinates in the coordinate system of the sorting device 30. The sorting device control unit 16 outputs a control signal including the converted extracted point coordinates to the sorting device 30.

[0031] Then, in step S150, the sorting device 30 moves the extraction unit 31 directly above the extraction point coordinates indicated in the control signal, and the extraction unit 31 uses the extraction point coordinates as the extraction point to extract bottles from the group of waste being transported on the belt conveyor 40. If the extraction unit 31 is formed using a robot hand 311, the extraction unit 31 sets the extraction point on the bottle as the gripping position of the robot hand 311 relative to the bottle and extracts the bottle by gripping it with the robot hand 311. If the extraction unit 31 is formed using a suction pad 312, the extraction unit 31 sets the extraction point on the bottle as the suction position of the suction pad 312 relative to the bottle and extracts the bottle by suction with the suction pad 312.

[0032] <Operation of the container sorting system> Figure 6 shows an example of the operation of the container sorting system of Embodiment 1 of this disclosure.

[0033] As shown in Figure 6, the image recognition unit 12 assigns feature information to the bottle image BI, including label information LA indicating the bottle type and contour information CO indicating the bottle region (step S105). Bottle types are distinguished, for example, as champagne bottles, wine bottles, PET bottles, and other bottles. Furthermore, with the long side of the rectangular waste group image as the X-axis and the short side as the Y-axis, the contour information CO is formed by a plurality of coordinate points (x0,y0), (x1,y1), ..., (xn,yn). In other words, the region enclosed by lines connecting the plurality of coordinate points (x0,y0), (x1,y1), ..., (xn,yn) that form the contour information CO is the bottle region.

[0034] The area centroid calculation unit 13 calculates the area centroid coordinates AC(Xa,Ya) in the bottle region based on the contour information CO (step S110).

[0035] The extraction point coordinate calculation unit 14 calculates the bottle length L, which is the total length of the bottle region in the longitudinal direction, based on the contour information CO (step S120).

[0036] Furthermore, the extraction point coordinate calculation unit 14 determines the bottom BP of the bottle based on the bottle image BI and calculates the weight centroid coordinate WC(Xw,Yw) by moving the area centroid coordinate AC(Xa,Ya) by a specified amount Δl in the direction of the bottom BP of the bottle (step S135). The extraction point coordinate calculation unit 14 determines the bottom BP of the bottle using, for example, a trained model stored in the trained model storage unit 15.

[0037] In the case of a PET bottle, the difference between the thickness of the bottom and the thickness of the rest of the bottle is small, so the center of gravity is almost the same as the center of gravity of the area. Therefore, when the bottle type is a PET bottle, the extraction point coordinate calculation unit 14 sets the area center of gravity coordinates to the extraction point coordinates (step S115: Yes, step S130). In other words, when the bottle type is a PET bottle, the extraction point coordinate calculation unit 14 sets the specified amount Δl to 0.

[0038] Furthermore, when the bottle length L is short, the center of gravity is almost the same as the center of area. Therefore, when the bottle length L is less than the threshold TH1, the extraction point coordinate calculation unit 14 sets the center of area coordinates to the extraction point coordinates (step S125: Yes, step S130). In other words, when the bottle length L is less than the threshold TH1, the extraction point coordinate calculation unit 14 sets the specified amount Δl to 0.

[0039] Furthermore, when calculating the weight centroid coordinates (step S135), the extraction point coordinate calculation unit 14 controls the specified amount Δl based on the bottle type. Figure 7 shows an example of a specified amount determination table for Embodiment 1 of this disclosure. As shown in Figure 7, the specified amount determination table TA has bottle types and specified amounts Δl set in correspondence with each other. The extraction point coordinate calculation unit 14 has a specified amount determination table TA and controls the specified amount Δl according to the specified amount determination table TA based on the bottle type. In the specified amount determination table TA, the specified amount Δl is set as a percentage [%] of the bottle length L.

[0040] Generally, the bottom thickness of a champagne bottle is greater than that of a wine bottle. Furthermore, the bottom thickness of both champagne bottles and wine bottles is greater than that of other bottles. In other words, the difference between the area centroid and the weight centroid is greater for champagne bottles than for wine bottles. Furthermore, the difference between the area centroid and the weight centroid is greater for wine bottles than for other bottles. Therefore, when the bottle type is a champagne bottle, the extraction point coordinate calculation unit 14 sets the specified amount Δl to 6% of the bottle length L according to the specified amount determination table TA. Furthermore, when the bottle type is a wine bottle, the extraction point coordinate calculation unit 14 sets the specified amount Δl to 4% of the bottle length L according to the specified amount determination table TA. Furthermore, when the bottle type is any other bottle, the extraction point coordinate calculation unit 14 sets the specified amount Δl to 2% of the bottle length L according to the specified amount determination table TA.

[0041] In this way, the extraction point coordinate calculation unit 14 controls a predetermined amount Δl based on the bottle type and bottle length L.

[0042] The above describes Example 1.

[0043] [Example 2] The trained model storage unit 15 is implemented as hardware, for example, by memory or storage. The image processing unit 11 and the sorting device control unit 16 are implemented as hardware, for example, by a processor such as a CPU (Central Processing Unit), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), or ASIC (Application Specific Integrated Circuit).

[0044] Furthermore, all or part of the processes described above in the image processing unit 11 and the sorting device control unit 16 may be implemented by having a processor execute a program corresponding to each process. For example, the program corresponding to each process described above may be stored in the memory of the control device 10, and the program may be read from memory and executed by the processor. Alternatively, the program may be stored in a program server connected to the control device 10 via any network and downloaded from the program server to the control device 10 for execution, or it may be stored in a recording medium readable by the control device 10 and read from that recording medium for execution. Recording media readable by the control device 10 include, for example, portable storage media such as memory cards, USB memory, SD cards, flexible disks, magneto-optical disks, CD-ROMs, and DVDs.

[0045] The above describes Example 2.

[0046] As described above, the container sorting system of this disclosure (container sorting system 1 of the embodiment) comprises a camera (camera 20 of the embodiment), a recognition unit (image recognition unit 12 of the embodiment), a first calculation unit (area centroid calculation unit 13 of the embodiment), a second calculation unit (extraction point coordinate calculation unit 14 of the embodiment), and an extraction unit (extraction unit 31 of the embodiment). The camera captures an image of the waste group on a transport path (belt conveyor 40 of the embodiment) on which the waste group is transported. The recognition unit recognizes the area and type of containers included in the waste group based on the image captured by the camera. The first calculation unit calculates the first coordinate (area centroid coordinate of the embodiment), which is the coordinate of the area centroid of the container in the image captured by the camera. The second calculation unit calculates the second coordinate (weight centroid coordinate of the embodiment), which is the coordinate obtained by moving a specified amount from the first coordinate toward the bottom of the container. The extraction unit uses the second coordinate as the extraction point for extracting containers and extracts containers from the waste group being transported on the transport path.

[0047] For example, the recognition unit uses a trained model to recognize the location and type of the container.

[0048] This allows the extraction point of the container to be set at the center of gravity of the container's weight, enabling the container to be transported without tilting after extraction. Furthermore, since the second coordinate is calculated based on the image captured by the camera, the second coordinate can be calculated without using measuring instruments or sensors to measure the weight of the container. In addition, because the container can be transported without tilting after extraction, the height to which the container needs to be lifted in the +Z direction can be reduced compared to when the container is tilted.

[0049] Furthermore, the second calculation unit controls the specified amount based on the type of container.

[0050] This allows the container to be transported without tilting, regardless of the type of container used.

[0051] Furthermore, the second calculation unit controls a predetermined amount based on the length of the container in the longitudinal direction.

[0052] This allows the container to be transported without tilting, regardless of its length in the longitudinal direction. [Explanation of symbols]

[0053] 1. Container sorting system 10 Control device 11 Image Processing Unit 12 Image Recognition Unit 13 Area center of gravity calculation part 14. Extraction point coordinate calculation unit 15. Pre-trained model memory 16. Sorting device control unit 20 cameras 30 sorting device 31 Extraction part

Claims

1. A first calculation unit calculates a first coordinate which is the coordinate of the area centroid of the container in an image taken by a camera that takes an image of the waste group, in a transport path through which the waste group including the container is transported. A second calculation unit calculates a second coordinate, which is the coordinate of the extraction point when the container is extracted from the group of waste being transported along the transport path, by moving a specified amount from the first coordinate toward the bottom of the container. A control device equipped with the following.

2. Based on the aforementioned image, a recognition unit recognizes the location and type of containers included in the waste group. The control device according to claim 1, further comprising:

3. The recognition unit recognizes the existence region and the type using a trained model. The control device according to claim 2.

4. The second calculation unit controls the specified amount based on the type. The control device according to claim 2.

5. The second calculation unit controls the specified amount based on the length of the container in the longitudinal direction. The control device according to claim 1.

6. The second calculation unit calculates the second coordinate based on the predetermined amount for each type recognized by the recognition unit. The control device according to claim 2.

Images