Valuable material sorting system and method for sorting valuable material

Abstract

To provide a technique capable of suppressing false detection in a valuable material sorting system.SOLUTION: A valuable material sorting system 1 includes: imaging means 4 for capturing an image of a sorting target object Ob on a conveyance path; a plurality of suction nozzles nx; and an information processing apparatus including a trained model capable of detecting foreign matter Cx in the sorting target object Ob from imaging data. The system further includes light-shielding means 9, and the information processing apparatus is capable of controlling operations of the suction nozzles nx on the basis of detection results of the trained model. Thus, the reliability of the detection results by the trained model can be ensured.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a valuable material sorting system and a valuable material sorting method for sorting various foreign objects and valuable materials. [Background technology]

[0002] At resource recycling plants that recover valuable materials from various types of waste, the purity of the recovered valuable materials is increased by sorting and removing foreign matter. For example, at glass material recycling plants, various foreign matter (e.g., resin, non-magnetic metal pieces, etc.) is sorted from glass scraps made from crushed solar panels, which are waste materials, to recover valuable glass materials. During the sorting process, for example, multiple workers are stationed along the glass scrap transport route, and the workers visually detect various foreign matter, and each worker manually operates a suction device to remove the foreign matter.

[0003] This manual sorting work requires multiple workers and is quite a heavy workload, so it is desirable to mechanize and automate the sorting work as much as possible.

[0004] Patent Document 1 (Japanese Patent Laid-Open Publication No. 62-177436) discloses a transmittance inspection device that is equipped with a light source, a photodetector, and a removal means (air jet) on the conveying path and is used to detect and remove opaque materials, such as foreign matter such as stone or ceramics, that are mixed in with glass fragments. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 177436 / 1983 Summary of the Invention [Problem to be solved by the invention]

[0006] At resource recycling plants that separate and recover valuable materials such as glass materials, foreign materials are gradually separated and removed from waste containing a mixture of foreign materials and valuable materials, ultimately increasing the purity of the valuable materials.

[0007] When optical detection methods such as various cameras are used to mechanize and automate the process of sorting and removing foreign objects, the shading, color, and other properties of the objects in the detection results will vary depending on the differences in various properties (individual differences) such as the material and size of the objects themselves, or the degree of scattering and reflection pattern of the light irradiated onto the objects.

[0008] The conventional device described in Patent Document 1 determines whether an object is valuable or foreign matter based on its transmittance, but if there are variations in shading or color among the same type of object, it may be detected as a foreign matter even though it is actually valuable (a false detection).

[0009] In view of the above, an object of the present invention is to provide a valuable resource sorting system and a valuable resource sorting method that can reduce false detections. [Means for solving the problem]

[0010] In order to solve the above problems, the first invention of the present application includes an imaging means for imaging a sorting target object on a conveyance path, which is a mixture of transparent valuables and opaque foreign objects, and acquiring imaging data of the imaging range; A valuable material sorting system that detects and removes foreign matter from the objects to be sorted, comprising a plurality of suction nozzles arranged downstream of the imaging means and an information processing device that is communicatively connected to both the suction nozzles and the imaging means, wherein the information processing device has a trained model that has been machine-learned to enable detection of the presence of foreign matter contained in the objects to be sorted when the imaging data is input, and the information processing device is capable of controlling the operation of the suction nozzles based on the detection results of the trained model.

[0011] The second invention of the present application is the valuables sorting system described in the first invention, characterized in that when the imaging data is input, the trained model can detect the presence of the foreign matter contained in the sorting object and estimate identification information that identifies the foreign matter, and the identification information includes area information that indicates the range that the foreign matter occupies on the transport path.

[0012] The third invention of the present application is the valuables sorting system described in the first invention, characterized in that when the imaging data is input, the trained model can detect the presence of the foreign object contained in the sorting object and estimate identification information that identifies the foreign object, and the identification information includes at least one of type information of the foreign object, area information indicating the range that the foreign object occupies on the conveying path, and reliability information that indicates the accuracy of the foreign object detection result.

[0013] The fourth invention of the present application is a valuable resource sorting system of the third invention, characterized in that, when the foreign object is detected, the operation of the removal means for the foreign object is postponed under any of the following conditions: the type information of the foreign object corresponds to a predetermined type, the size of the area information of the foreign object is below a predetermined area lower limit, or the reliability information of the foreign object is below a predetermined reliability lower limit.

[0014] A fifth invention of the present application is a valuables sorting system according to any one of the first to fourth inventions, characterized in that, when the imaging data is input, the trained model is able to detect the presence of the foreign matter contained in the sorting object and estimate identification information that identifies the foreign matter, the identification information including at least position information indicating the position of the foreign matter on the transport path, and the information processing device selects the suction nozzle corresponding to the position information as the object to be operated.

[0015] The sixth invention of the present application is a method for sorting valuable materials, which uses multiple suction nozzles to remove foreign matter from a sorting object that contains a mixture of valuable materials and foreign matter, and includes: a) a step of capturing an image of the sorting object on a transport path to obtain image data; b) a step of using a trained model to detect the foreign matter contained in the image data; and c) a step of selecting one of the suction nozzles to be operated based on the detection results of the trained model. [Effects of the Invention]

[0016] According to the present invention, a valuable resource sorting system, a method for creating a trained model, and a program can be provided that can suppress false detection of sorting objects.

[0017] In particular, according to the first invention, by using a trained model that has been machine-learned, it is possible to grasp the type of object, including variations in shading, color, etc., thereby reducing false detections.

[0018] In particular, according to the second aspect of the present invention, the polarization axis of the light irradiating the object to be sorted is perpendicular to the polarization axis of the light that can be incident on the imaging means from the object to be sorted. This allows imaging of only the diffusely reflected light from the object to be sorted. This reduces the reflection of the internal light source and allows obtaining good imaging data that reflects the unique color of the object to be sorted, further ensuring the reliability of the detection results.

[0019] In particular, according to the second and third inventions, the sorting system can be appropriately controlled based not only on the presence or absence of foreign matter but also on quantitative or qualitative identification information that identifies each foreign matter. Specifically, the removal means can be activated in accordance with the size, type, or reliability of the detection result of the foreign matter. Therefore, excessive removal of valuable materials can be prevented (reducing yield reduction).

[0020] In particular, according to the fifth aspect of the invention, foreign objects located anywhere on the transport path can be sucked up by the suction nozzle in principle, which makes the configuration relatively simple and does not require complex control compared to devices such as mobile robot arms. [Brief explanation of the drawings]

[0021] [Figure 1] FIG. 1 is a side view showing an outline of a sorting system. [Figure 2] FIG. 1 is a top view showing an outline of a sorting system. [Figure 3] FIG. 1 is a schematic diagram showing the placement of cameras and LED lights inside the hood of the sorting system. [Figure 4] FIG. 4 is a diagram showing the arrangement of nozzles in a suction unit. [Figure 5] FIG. 2 is a block diagram showing electrical connections of the information processing device. [Figure 6] FIG. 2 is a diagram showing setting registration values ​​stored in a storage unit of the information processing device. [Figure 7] 1 is a flowchart showing the flow of the process of creating a trained model. [Figure 8] 10 is a flowchart showing the flow of valuable resource sorting processing in the sorting system. [Figure 9] 10 is a flowchart showing the flow of a suction control process in the sorting system. [Figure 10] 1A and 1B are diagrams showing examples of foreign matter and a suction nozzle. [Figure 11] 1A and 1B are diagrams showing examples of foreign matter and a suction nozzle. [Figure 12] 1A and 1B are diagrams showing examples of foreign matter and a suction nozzle. [Figure 13] 1A and 1B are diagrams showing examples of foreign matter and a suction nozzle. [Figure 14] 1A and 1B are diagrams showing examples of foreign matter and a suction nozzle. [Figure 15] FIG. 10 is a diagram illustrating an example of a plurality of foreign objects and a foreign object suction task. [Figure 16] FIG. 10 is a diagram illustrating an example of a plurality of foreign objects and a foreign object suction task. [Figure 17] 10 is a flowchart showing the flow of a conflict time control process (without exception control). [Figure 18] FIG. 10 is a diagram illustrating an example of a plurality of foreign objects and a foreign object suction task. [Figure 19]10 is a flowchart showing the flow of a conflict time control process (with exception control). [Figure 20] FIG. 10 is a diagram illustrating an example of a plurality of foreign objects and a foreign object suction task. [Figure 21] FIG. 10 is a diagram showing an example of a plurality of foreign objects. [Figure 22] FIG. 22 is a diagram showing a foreign object suction task in the example of FIG. 21. [Figure 23] 1A and 1B are diagrams showing examples of foreign matter and a suction nozzle. [Figure 24] 1A and 1B are diagrams showing examples of foreign matter and a suction nozzle. DETAILED DESCRIPTION OF THE INVENTION

[0022] <1. Overall configuration of the valuables sorting system> The configuration of a sorting system 1, which is a valuable resource sorting system according to one embodiment of the present invention, will be described below with reference to FIGS. 1 and 2. FIG. 1 is a side view showing an overview of the sorting system 1. FIG. 2 is a top view showing an overview of the sorting system 1. The sorting system 1 is an apparatus for detecting and removing foreign matter from sorting objects Ob containing a mixture of valuable resources and foreign matter, and recovering the valuable resources. The following describes a case where the sorting objects Ob are crushed solar panels, the valuable resources are transparent glass material Gl, and the foreign matter is an opaque object (foreign matter Cx).

[0023] The sorting system 1 includes a conveyor 2, a feed chute 3, a camera 4, a suction unit 5, and an information processing device 6.

[0024] The conveyor 2 is a transport means for transporting the objects to be sorted Ob. The transport surface of the conveyor 2 moves from one end of the conveyor 2 to the other end. As a result, the objects to be sorted placed on the conveyor 2 are transported from one end of the conveyor 2 to the other end. In other words, the transport surface of the conveyor 2 forms a transport path for the objects to be sorted. Therefore, the direction from one end of the conveyor 2 to the other end is the "transport direction." In the following, one end of the conveyor 2 will be referred to as the "upstream side," and the other end will be referred to as the "downstream side."

[0025] A valuable resource recovery hopper 31 is disposed below the other end of the conveyor 2. As a result, the objects to be sorted Ob that have moved from one end to the other on the conveyor 2 drop from the other end of the conveyor 2 and are recovered into the valuable resource recovery hopper 31.

[0026] The input chute 3 is located near one end of the conveyor 2 and inputs the objects to be sorted Ob onto the conveyance surface of the conveyor 2. That is, the input chute 3 places the objects to be sorted Ob on the conveyance path.

[0027] Above (vertically above) the conveyor 2, there are arranged, in order from the upstream side, a camera 4 serving as an imaging means and a suction unit 5. The camera 4 and the suction unit 5 are each connected to an information processing device 6 via a wired or wireless network so as to be able to communicate with each other.

[0028] The camera 4 is an imaging unit that captures an image of the sorting target object Ob moving on the transport path to obtain image data. The camera 4 transfers the image data to the information processing device 6.

[0029] The suction unit 5 is located downstream of the camera 4 and above the conveyor 2, which is the transport path. The suction unit 5 includes multiple first suction nozzles Nx and multiple second suction nozzles nx. The first suction nozzles Nx and the second suction nozzles nx have different apertures. The multiple suction nozzles Nx and nx of the suction unit 5 are each activated / deactivated by the information processing device 6.

[0030] More specifically, the suction unit 5 includes solenoid valves 7 that control the operation of each of the suction nozzles Nx and nx. The solenoid valves 7 are physically connected to a PLC (Programmable Logic Controller) 8. The PLC 8 is connected to the information processing device 6 so as to be able to communicate with each other.

[0031] This allows various instructions sent from the information processing device 6 to be transmitted to the solenoid valve 7 via the PLC 8, and the operation / stop of each suction nozzle Nx, nx can be controlled by controlling the opening and closing of the solenoid valve 7. In other words, the suction unit 5 equipped with the solenoid valve 7 is connected to the information processing device 6 via the PLC 8 so as to be able to communicate with it.

[0032] The information processing device 6 is, for example, a computer (PC terminal) including a processor such as a CPU or a GPU, a memory such as a RAM, and a storage medium such as a hard disk drive or an SSD.

[0033] Based on the imaging data, the information processing device 6 activates the suction nozzles Nx, nx of the suction unit 5. The suction nozzles Nx, nx suck a part of the selection target object Ob being transported on the conveyor 2 and remove it from the transport path.

[0034] In this sorting system 1, sorting objects Ob are loaded onto a conveyor 2 from a loading chute 3 and transported along a transport path on the conveyor 2. Then, the sorting objects Ob transported on the conveyor 2 are photographed by a camera 4, and then, based on the image data, foreign matter Cx is sucked and removed in a suction unit 5 using suction nozzles Nx, nx.

[0035] The sorting target Ob that has undergone this sorting process has a lower proportion of foreign matter Cx mixed in compared to the sorting target Ob before sorting, i.e., the purity of the valuable material Gl is higher. The sorting target Ob after removal of the foreign matter Cx is collected into the valuable material collection hopper 31.

[0036] <2. About the imaging environment> When photographing the selection object Ob with the camera 4, it is necessary to provide light blocking and appropriate lighting in order to perform a more accurate analysis of the photographed data. Details of the photographing environment will be described below with reference to FIGS. 1 to 3.

[0037] As shown in FIG. 1, the camera 4 and an imaging range 10 below the camera 4 are covered by a hood 9. The imaging range 10 is a predetermined range of the conveying surface 22 of the conveyor 2 where the camera 4 takes images. In this embodiment, the imaging range 10 is almost the entire area of ​​the conveying surface 22 of the conveyor 2 that is covered by the hood 9. However, the imaging range 10 may be smaller than this range as long as it covers the entire width of the conveying path. The hood 9 is a light-shielding means made of a light-shielding material. The hood 9 covers a portion of the conveying surface 22 of the conveyor 2 that includes the imaging range 10.

[0038] As shown in FIGS. 1 and 2, the sorting system 1 has two LED lights 11 inside the hood 9. The two LED lights 11 are internal light sources that irradiate light toward the imaging range 10 of the camera 4. The hood 9 houses the two LED lights 11 and the camera 4. The hood 9 and the conveyor 2 form a shooting environment that blocks external ambient light. Therefore, the shooting environment inside the hood 9 can be adjusted by the LED lights 11 to achieve desired imaging conditions, such as desired illuminance and color temperature.

[0039] A scraper 12 is provided near the entrance (upstream end) of the hood 9. The scraper 12 is a means for leveling the objects to be sorted Ob. The scraper 12 is, for example, a plate-shaped elastic member whose upper part is fixed and whose lower end is disposed at a distance from the conveying surface 22 of the conveyor 2. Portions of the objects to be sorted Ob transported from the upstream side of the scraper 12 that have piled up and are higher than the lower end of the scraper 12 come into contact with the lower end of the scraper 12 and are leveled to a predetermined height. This prevents various problems, such as a deterioration in detection accuracy due to the objects to be sorted Ob overlapping each other.

[0040] 3 is a schematic side view showing the arrangement of the conveyor 2, camera 4, and LED light 11 inside the hood 9. As described above, within the hood 9, almost the entire conveying surface 22 of the conveyor 2 is included in the imaging range 10.

[0041] 1 to 3, each of the two LED lights 11 extends along the width direction of the conveyor 2. This allows the LED lights 11 to emit illumination light onto the imaging range 10 uniformly in the width direction.

[0042] The two LED lights 11 are arranged at an interval in the conveying direction. One of the LED lights 11 is arranged upstream of the camera 4 in the conveying direction and emits light downward and downstream in the conveying direction. The other LED light 11 is arranged downstream of the camera 4 in the conveying direction and emits light downward and upstream in the conveying direction.

[0043] Specifically, as shown in Fig. 3, the two LED lights 11 are installed below (vertically below) the camera 4, equidistant in the conveying direction from the central axis 40 of the camera 4. The orientation of the LED lights 11 is determined so that the angle θi formed between the optical axis 13 of each LED light 11 and the conveying surface 22 of the conveyor 2 is an acute angle. The optical axis 13 of each LED light 11 extends perpendicular to the illumination surface 15 of the lighting body 14 of the LED light 11.

[0044] In this embodiment, the angle θi is approximately 25°. The angle θi is preferably 10° or more and 45° or less, and more preferably 20° or more and 30° or less. If the angle θi significantly exceeds 45°, the outline (edge) of the glass material Gl may be erroneously detected as the presence range of the foreign substance Cx due to shadows, or light may be scattered excessively, making it difficult to obtain accurate detection results.

[0045] As shown in Fig. 3, the LED lighting device 11 is provided with a first polarizing plate 16 on the irradiation surface 15 of the lighting device body 14. The first polarizing plate 16 is a first polarization means that polarizes the light emitted from the LED lighting device 11. Meanwhile, the camera 4 is provided with a second polarizing plate 44 on the lower light collecting surface 43 of the lens 42 provided below the camera body 41. The second polarizing plate 44 is a second polarization means that polarizes the light that enters the camera 4.

[0046] The first polarizer 16 and the second polarizer 44 each have an extremely fine (nano-level) slit structure formed by stretching a plastic such as PVA with an elongated molecular structure. This allows the first polarizer 16 and the second polarizer 44 to pass only light that vibrates parallel to the slits. In other words, the first polarizer 16 and the second polarizer 44 each have a polarization axis parallel to the slits. The polarization axis of the first polarizer 16 and the polarization axis of the second polarizer 44 are arranged in directions perpendicular to each other.

[0047] Light emitted from the LED light 11 and polarized by the first polarizer 16 (hereinafter referred to as "first polarized light") is irradiated onto the sorting objects Ob. Then, the first polarized light is reflected by the sorting objects Ob and the conveying surface 22 of the conveyor 2, and the reflected light is incident on the camera 4 via the second polarizer 44. Therefore, the camera 4 captures light (hereinafter referred to as "second polarized light") that is further polarized by the second polarizer 44 from the reflected light of the first polarized light.

[0048] In this case, the surface of the sorting object Ob does not only have flat areas but also has irregular surfaces with bumps. Therefore, part of the first polarized light incident on the sorting object Ob from the LED light 11 via the first polarizer 16 becomes specularly reflected light, part becomes diffusely reflected light, and part is absorbed. Here, the diffusely reflected light reflects the original color of the subject, the sorting object Ob. On the other hand, the specularly reflected light reflects the color of the LED light 11, which may cause glossiness in the imaging data and adversely affect detection accuracy.

[0049] Because the specularly reflected light has the same polarization axis as the incident light, the specularly reflected light of the first polarized wave does not pass through the second polarizing plate 44. Therefore, the specularly reflected light of the first polarized wave is not included in the second polarized wave. In other words, the image data captured by the camera 4 does not include the specularly reflected light.

[0050] On the other hand, the diffusely reflected light becomes unpolarized even if the incident light is a polarized first polarized wave, and therefore the component of the diffusely reflected light of the first polarized wave in the polarization direction of the second polarizer 44 is incident on the camera 4 as a second polarized wave. That is, the imaging data captured by the camera 4 includes the diffusely reflected light.

[0051] In this way, in this sorting system 1, by providing a first polarizing plate 16 and a second polarizing plate 44 on the light path consisting of the LED lighting 11, the sorting object Ob, and the camera 4, it is possible to suppress reflections of the LED lighting 11 and obtain good image data that reflects the unique color of the sorting object Ob.

[0052] Furthermore, a fluorescent lamp or an incandescent lamp can be used as the internal light source instead of the LED lamp 11. However, since the flicker of a fluorescent lamp or an incandescent lamp adversely affects the detection accuracy, the LED lamp 11, which does not cause such flicker, is more suitable.

[0053] Furthermore, if the color temperature of the internal light source is in a relatively low temperature range that emits warm colors, this may have a negative effect on detection accuracy. For this reason, it is preferable that the color temperature of the internal light source be around 6000K to 7000K (daylight color or daylight color) rather than around 2800K (warm warm color) of an incandescent lamp.

[0054] In this way, by arranging the camera 4, LED lighting 11, and imaging range 10 inside the light-blocking hood 9, disturbances caused by ambient light can be suppressed. Also, by keeping the positional relationship between the camera 4, LED lighting 11, and imaging range 10 within the above range, the formation of shadows and excessive scattering of light can be suppressed. Furthermore, by placing two polarizing plates 16, 44 on the optical path from the LED lighting 11 to the camera 4, it is possible to selectively capture diffuse reflected light from the sorting object Ob. This makes it possible to acquire imaging data suitable for accurately identifying the foreign matter Cx.

[0055] The camera 4 is, for example, a general visible light camera that uses visible light to capture an image of an object, and although one camera is provided in this embodiment, it may also be configured with a plurality of cameras.

[0056] <3. Configuration of the suction unit> Next, the configuration of the suction unit 5 will be described with reference to Figures 1 to 2 and 4. Figure 4 is a diagram showing the arrangement of the suction nozzles Nx and nx in the suction unit 5. Figure 4 shows the positions of the nozzle openings of the first suction nozzle Nx and the second suction nozzle nx as viewed from above.

[0057] The suction unit 5 is a pneumatic removal means capable of removing a portion of the objects to be sorted Ob on the transport path. Specifically, the suction unit 5 has a plurality of first suction nozzles Nx and a plurality of second suction nozzles nx. The plurality of first suction nozzles Nx all have the same aperture. Also, the plurality of second suction nozzles nx all have the same aperture. The plurality of first suction nozzles Nx and the second suction nozzles nx have different apertures. Specifically, the aperture of the first suction nozzles Nx is larger than the aperture of the second suction nozzles nx.

[0058] As shown in FIG. 1, the suction unit 5 includes a compressor 25 , a duct 26 , a plurality of hoses 27 , a chamber 28 , a discharge hose 29 , and a foreign object hopper 30 .

[0059] The compressor 25 is a compressed air supply source. The duct 26 is a supply pipe that sends the compressed air supplied from the compressor 25 to the chamber 28. The multiple hoses 27 are pipes that connect the multiple suction nozzles Nx, nx to the chamber 28. A solenoid valve 7 is provided at the connection point between each hose 27 and the chamber 28. Each solenoid valve 7 controls communication between the connected hose 27 and the chamber 28. The discharge hose 29 is a pipe that transfers the collected foreign matter Cx from the bottom of the chamber 28 to the foreign matter hopper 30. The foreign matter hopper 30 is a container that stores the collected foreign matter Cx.

[0060] Compressor 25 introduces compressed air into chamber 28 via duct 26 and a regulator (not shown). This generates an airflow that flows through duct 26, chamber 28, and discharge hose 29 toward foreign object hopper 30. At this time, negative pressure is generated in chamber 28 due to the intake of the compressed air flow.

[0061] A suction control unit 62 of the information processing device 6, which will be described later, inputs instructions regarding the suction operation of each of the suction nozzles Nx, nx to the PLC 8. The PLC 8 sends signals corresponding to these instructions to the multiple solenoid valves 7 connected to each of the multiple suction nozzles Nx, nx, controlling the opening and closing of the solenoid valves 7. When the solenoid valves 7 are open, the connected suction nozzles Nx, nx are in an operating state (suction state), and when the solenoid valves 7 are closed, the connected suction nozzles Nx, nx are in a stopped state.

[0062] When the solenoid valve 7 is opened while the chamber 28 is under negative pressure due to the operation of the compressor 25, the air inside the hose 27 and the suction nozzles Nx, nx connected to the solenoid valve 7 is sucked into the chamber 28. As a result, the objects to be sorted Ob near the nozzle openings of the suction nozzles Nx, nx are sucked in and collected into the foreign object hopper 30 via the suction nozzles Nx, nx, the hose 27, the chamber 28, and the discharge hose 29.

[0063] Once the compressor 25 is operating, it can store a predetermined amount of compressed air as internal air in a tank (not shown). Therefore, the compressor 25 operates intermittently only when the internal air is used up or when the internal air falls below the required amount.

[0064] The suction nozzles Nx, nx are connected to a chamber 28 via a hose 27. The foreign matter Cx (including a portion of the glass material Gl that is sucked together with the foreign matter Cx) that is sucked into the chamber 28 is guided to a foreign matter hopper 30 via a discharge hose 29.

[0065] In this way, by sorting and removing the foreign matter Cx from the objects to be sorted Ob, the purity of the valuable material (glass material Gl) in the objects to be sorted Ob is increased, and the valuable material is collected as a final product in a valuable material recovery hopper 31 on the downstream side of the conveyor 2. Note that in this embodiment, as a method for collecting the glass material Gl, the valuable material recovery hopper 31 is provided on the other end side of the conveyor 2, but the means for collecting the glass material Gl, which is a valuable material, is not limited to this.

[0066] As shown in FIGS. 2 and 4, the suction unit 5 has nozzle rows 51 and 52 for each diameter of the suction nozzles Nx and nx. Specifically, the suction unit 5 has a first nozzle row 51 and a second nozzle row 52. The first nozzle row 51 is made up of a plurality of first suction nozzles Nx arranged across the entire width of the conveyance path on the conveyor 2. The second nozzle row 52 is made up of a plurality of second suction nozzles nx arranged across the entire width of the conveyance path on the conveyor 2. In this embodiment, the suction unit 5 has 10 first suction nozzles Nx and 20 second suction nozzles nx. The number of each suction nozzle Nx and nx can be increased or decreased as appropriate depending on the layout of the factory equipment in which the sorting system 1 is installed, etc.

[0067] In this embodiment, the second suction nozzles nx in the second nozzle row 52 are arranged in a zigzag pattern so that the positions in the conveying direction of adjacent second suction nozzles nx in the width direction of the conveyor 2 are different. However, in the second nozzle row 52, ​​similar to the first nozzle row 51, all of the second suction nozzles nx may be arranged at the same position in the conveying direction.

[0068] In this embodiment, the first nozzle row 51 is disposed upstream of the second nozzle row 52. That is, the first suction nozzle Nx, which has a larger diameter, is disposed upstream of the second suction nozzle nx, which has a smaller diameter. The reason for this will be described later.

[0069] As described above, each of the multiple suction nozzles Nx, nx is equipped with a solenoid valve 7 connected to the PLC 8. In this embodiment, the suction nozzles Nx, nx are fixed and do not move horizontally. That is, the only control over the suction unit 5 during the foreign matter removal process is the opening and closing control of the solenoid valve 7. This configuration has the advantage that it does not require overly complex control commands and that it is easy to add more suction nozzles Nx, nx at low cost.

[0070] However, although not shown, the suction nozzles Nx, nx may be attached to the tip of a robot arm that can be controlled to move horizontally (particularly in the width direction), or may be attached to one end of a piston rod-like mechanism that can be controlled to move vertically.

[0071] In this embodiment, the suction nozzles Nx, nx of the suction unit 5 have two different diameters. However, the suction nozzles of the suction unit 5 may have one type of diameter, or three or more types of diameters. For example, multiple suction nozzles having three types of diameters, large, medium, and small, may be arranged from the upstream side to the downstream side of the conveyor 2 in descending order of diameter.

[0072] 4, the ten first suction nozzles Nx are each given the individual reference numerals N1 to N10, and similarly, the twenty second suction nozzles nx are each given the individual reference numerals n1 to n20.

[0073] As shown in Figure 4, the diameter of the first suction nozzle Nx is the inner diameter of the nozzle opening. Therefore, the first suction nozzle Nx cannot suck in sorting objects Ob that are larger than the diameter. Similarly, the diameter of the second suction nozzle nx is the inner diameter of the nozzle opening. Therefore, the second suction nozzle nx cannot suck in sorting objects Ob that are larger than the diameter.

[0074] In the foreign matter removal process, which suction nozzle Nx, nx is selected is determined based on whether the foreign matter Cx to be removed passes through the suction area of ​​the suction nozzle Nx, nx. Hereinafter, the area that can be sucked by the first suction nozzle Nx will be referred to as the suction area Vx, and the area that can be sucked by the second suction nozzle nx will be referred to as the suction area vx.

[0075] In Figure 4, the suction area V1 of the first suction nozzle N1 and the suction area v1 of the second suction nozzle n1 are shown as shaded areas. The areas that can actually be suctioned by each suction nozzle Nx and nx are larger than these suction areas Vx and vx. Setting the suction areas Vx and vx in a rectangular shape makes it easier to calculate the timing for suctioning the foreign object Cx.

[0076] It is preferable that the widthwise spacing between the first suction nozzles Nx in the first nozzle row 51 is set so that there is no area in the width direction of the conveyor 2 where there is no suction area Vx. It is also preferable that the widthwise spacing between the second suction nozzles nx adjacent in the width direction in the second nozzle row 52 is set so that there is no area in the width direction of the conveyor 2 where there is no suction area vx.

[0077] Furthermore, in the first nozzle row 51, the suction regions Vx of the first suction nozzles Nx adjacent in the width direction may overlap each other at their widthwise ends. Similarly, in the second nozzle row 52, ​​the suction regions vx of the second suction nozzles nx adjacent in the width direction may overlap each other at their widthwise ends.

[0078] However, if the suction areas Vx, vx overlap in the conveying direction, there is a risk that one of the suction nozzles Nx, nx will interfere with an object within the suction range of the other suction nozzle Nx, nx, causing the position of the selected object Ob to be shifted.

[0079] For this reason, it is preferable to arrange the suction nozzles Nx and nx at a sufficient distance in the transport direction so that the suction areas Vx and vx do not overlap, at least in the transport direction. That is, the suction nozzles Nx and nx are arranged so that the suction area Vx of the first suction nozzle Nx does not overlap with the suction area vx of the second suction nozzle nx, and so that the suction area vx of the second suction nozzle nx arranged upstream does not overlap with the suction area vx of the second suction nozzle nx arranged downstream. Furthermore, when the first suction nozzles Nx are arranged in multiple rows in the transport direction, the suction nozzles Nx and nx are arranged so that the suction areas Vx of the first suction nozzles Nx do not overlap.

[0080] As described above, the multiple first suction nozzles Nx are arranged so that the entire width of the conveyor 2 is included in the suction area Vx of any one of the first suction nozzles Nx. In addition, the multiple second suction nozzles nx are arranged so that the entire width of the conveyor 2 is included in the suction area vx of any one of the second suction nozzles nx.

[0081] In Figure 4, the area obtained by extending the suction area Vx of each first suction nozzle Nx (N1 to N10) in the conveyance direction is shown as a virtual first lane Lx (L1 to L10) on the conveyor 2. Also, the area obtained by extending the suction area vx of each second suction nozzle nx (n1 to n20) in the conveyance direction is shown as a virtual second lane lx (l1 to l20) on the conveyor 2. Any point on the conveyor 2 is located on either of the first lanes Lx and also on either of the second lanes lx.

[0082] The sorting objects Ob are transported in the transport direction by the conveyor 2 while maintaining their widthwise position. That is, the sorting objects Ob are transported along the same first lane Lx and the same second lane lx without changing their widthwise position during transport. They eventually reach the suction area Vx of the first suction nozzle Nx corresponding to the first lane Lx, and (if there is no suction by the first suction nozzle Nx) the suction area vx of the second suction nozzle nx corresponding to the second lane lx. Therefore, by detecting the widthwise and transportwise positions of the foreign objects Cx upstream of the suction unit 5, the corresponding first suction nozzle Nx and second suction nozzle nx can be identified, and the foreign objects Cx can be sucked and removed using the suction nozzles Nx and nx located in the appropriate positions.

[0083] <4. Configuration of information processing device> Fig. 5 is a block diagram showing the electrical connections of the information processing device 6. As shown in Fig. 5, the information processing device 6 has a detection unit 61, a suction control unit 62, and a storage unit 63. The detection unit 61 and the suction control unit 62 are arithmetic processing units that are realized by temporarily reading a computer program stored in a storage medium into a memory and having a processor perform arithmetic processing based on the computer program and various data.

[0084] The detection unit 61 detects and identifies a foreign object Cx based on the image data captured by the camera 4. The detection unit 61 includes a trained model M and a parameter calculation unit 611.

[0085] Specifically, the detection unit 61 receives imaging data of the imaging range 10 as an input signal from the camera 4. In this embodiment, the imaging range 10 of the imaging data is a single region, but to improve processing speed, the imaging data may be divided into a plurality of predetermined regions and sent to the information processing device 6, and the information processing device 6 may then combine and use the regions.

[0086] The detection unit 61 then inputs the imaging data to the trained model M. With the imaging data input, the trained model M detects a foreign object Cx from the sorting object Ob and estimates identification information that identifies the foreign object Cx. The parameter calculation unit 611 calculates various parameters by performing predetermined calculations based on the presence or absence of the foreign object Cx and the identification information of the foreign object Cx output by the trained model M. Hereinafter, the identification information estimated by the trained model M will be referred to as "original identification information D1," and the various parameters calculated by the parameter calculation unit will be referred to as "secondary identification information D2."

[0087] The suction control unit 62 controls the operation of the solenoid valve 7 via the PLC 8 based on the presence or absence of the foreign matter Cx, the identification information of the foreign matter Cx, and various parameters obtained by the detection unit 61. This controls the suction operation of the suction nozzles Nx, nx.

[0088] The memory unit 63 stores various pre-set parameters, the original identification information D1 of each foreign body Cx output by the trained model M, the secondary identification information D2 calculated by the parameter calculation unit 611, various calculation formulas used in the parameter calculation unit 611, and various setting registration values ​​and condition parameters used for suction control in the suction control unit 62.

[0089] 2, the information processing device 6 is communicably connected to a compressor / conveyor control panel 20 that controls the operation of the compressor 25 and the conveyor 2. The values ​​set in the compressor / conveyor control panel 20 are stored in a memory unit 63 of the information processing device 6.

[0090] <4-1. About the detection unit / trained model M> The trained model M is machine-trained to detect the presence or absence of various foreign matter Cx from the sorting object Ob, and to estimate area information D11, position information (coordinates) D12, type information D13, and reliability information D14 as original identification information D1 relating to the characteristics of the detected foreign matter Cx. The machine learning method of the trained model M will be described later.

[0091] The area information D11 indicates the area occupied by the foreign object Cx on the conveyance path and includes at least information that serves as a basis for calculating the size (dimensions) of the foreign object Cx. The area information D11 is typically expressed as a graphic (hereinafter referred to as an "area graphic") defined in a predetermined shape so as to include at least the area occupied by the foreign object Cx on the conveyor 2. In this case, the geometric quantities of the area graphic (such as the distance between points included in the area graphic and the area of ​​the area graphic) are used as an approximation of the size of the foreign object Cx. In this embodiment, the area graphic is a bounding box defined by a rectangle circumscribing the foreign object Cx. In this case, the area information D11 includes the vertical width (the length of the sides parallel to the conveyance direction) and horizontal width (the length of the sides parallel to the width direction of the conveyor 2) of the bounding box. As a result, the diagonal length of the bounding box can be calculated as described below, and used as an approximation of the size of the foreign object Cx. The area information D11 may also include information indicating the height of the foreign object Cx.

[0092] The position information D12 is information indicating the position of the detected foreign matter Cx in the transport direction and width direction. The position information D12 is, for example, the center coordinates (center of gravity coordinates) of a bounding box, which is an area graphic.

[0093] The type information D13 is information indicating the type of the detected foreign object Cx. When the sorting object Ob is a crushed solar panel, the types of the foreign object Cx are pre-registered as labels such as plastic, washer, adhesive, etc. When the trained model M detects the foreign object Cx, it estimates which of these foreign object labels the foreign object Cx falls under and outputs the result as type information D13.

[0094] The reliability information D14 is information indicating the reliability of the estimation results, such as the area information D11, the location information D12, and the type information D13, estimated by the trained model M. Note that the reliability information D14 may be a calculation result based on these estimation results.

[0095] Specifically, the reliability information D14 may be a classification accuracy indicating the accuracy of the type information D13. Alternatively, the reliability information D14 may be the degree of fit of a bounding box to the contour of the foreign object Cx (which indicates the accuracy of the area information D11 and the position information D12). In this case, the reliability information D14 may be, for example, various index values ​​such as an Intersection over Union (IOU). Alternatively, the reliability information D14 may be, for example, a correlation coefficient indicating the degree of pattern match between the teacher image data and the imaging data. Alternatively, the reliability information D14 may be a parameter calculated comprehensively based on these multiple reliabilities.

[0096] In this embodiment, the reliability information D14 is calculated based on the classification accuracy of the type information D13 of the foreign object Cx and the IOU. Examples of erroneous type information D13 include a misclassification of one foreign object Cx with another (e.g., a false detection where plastic is mistakenly detected as a non-magnetic metal piece) and a false detection of a glass material Gl as some kind of foreign object Cx. In particular, when a glass material Gl is falsely detected as a foreign object Cx, the reliability information D14 tends to appear significantly low. Therefore, the lower the value of this reliability information D14, the higher the risk of a false detection where a glass material Gl (a valuable resource) is falsely detected as a foreign object Cx, and the reliability information D14 serves as an index of the risk of false detection.

[0097] The reliability information D14 is expressed quantitatively, and can be expressed, for example, as a numerical value, a percentage, or a probability. In this embodiment, the reliability information D14 is expressed as a numerical value between 0 and 1, and when the reliability information D14 is 1, the classification accuracy of the detection result is highest, and when the reliability information D14 is 0, the classification accuracy of the detection result is lowest.

[0098] The original identification information D1 estimated by the trained model M described above includes not only the presence or absence of a foreign substance Cx but also quantitative and qualitative information. Therefore, it is useful for appropriate control of the suction unit 5 and is used for suction control in the suction control unit 62 as appropriate.

[0099] On the other hand, the original identification information D1 becomes even more useful in appropriately controlling the suction unit 5 by processing it into secondary information (secondary identification information D2) in the parameter calculation unit 611. For this reason, the original identification information D1 is input to the parameter calculation unit 611, and the secondary identification information D2, which is the calculation result, is sent to the suction control unit 62 together with the original identification information D1.

[0100] <4-2. Detection section / parameter calculation section> The parameter calculation unit 611 calculates secondary identification information D2 by receiving input of each piece of information D11 to D14 of the original identification information D1 primarily estimated by the trained model M. Below, the secondary identification information D2 will be explained on the assumption that the trained model M has obtained area information D11 and position information D12 using a bounding box when detecting a foreign object Cx.

[0101] Prior to the parameter calculation unit 611 calculating the secondary identification information D2, various characteristic values ​​such as the density standard value and height standard value of the foreign matter Cx are stored in advance in the storage unit 63 for each foreign matter type. The parameter calculation unit 611 then calculates the desired secondary identification information D2 by substituting each value of the original identification information D1 primarily estimated by the trained model M into a predetermined calculation formula, and by appropriately retrieving the set registered values ​​and conditions 40 from the storage unit 63 as predetermined constants for each calculation formula and reflecting them in the calculation formula. Each calculation formula can be used alone or in combination with multiple calculation formulas.

[0102] In this embodiment, the secondary identification information D2 includes a diagonal length D21, area information D22, and weight information D23.

[0103] The diagonal length D21 is the length of the diagonal of the bounding box, calculated from the size of the bounding box included in the area information D11. The parameter calculation unit 611 squares the vertical and horizontal widths of the bounding box, respectively, to find their sum, and sets the square root of this sum as the diagonal length D21. Note that if the area indicated by the area information D11 has a shape other than a bounding box, the distance between the two most distant points in that area may be used as the diagonal length D21.

[0104] The diagonal length D21 is used as information indicating the size of the area information D11. If a suction nozzle Nx, nx with a diameter larger than at least the diagonal length D21 is used, the target foreign substance Cx can be sucked in, so the diagonal length D21 is used to properly select a suction nozzle Nx, nx (with the minimum necessary diameter) that matches the size of the foreign substance Cx.

[0105] The area information D22 is a value indicating the area of ​​the area information D11. The parameter calculation unit 611 sets the product of the vertical and horizontal widths of the bounding box included in the area information D11 as the area information D22. Note that if the area indicated by the area information D11 has a shape other than a bounding box, the area of ​​the area may be set as the area information D22.

[0106] The weight information D23 is a reference value related to the weight of the foreign matter Cx. The parameter calculation unit 611 multiplies the area information D22 by the height standard value and density standard value of various foreign matter Cx that are set and registered in advance, to obtain the weight information D23.

[0107] In this embodiment, a stereo camera is not used as the camera 4, and therefore, standard height values ​​of various foreign objects Cx that are set and registered in advance are used in calculating the weight information D23. However, a stereo camera may be provided separately in addition to a conventional camera as the camera 4. In that case, the measured height of the foreign object Cx may be used in calculating the weight information D23 instead of the standard height values.

[0108] Furthermore, in calculating the weight information D23, a standard weight value of the foreign matter Cx that is set and registered in advance for each foreign matter type may be used instead of the standard height value or standard density value. In this way, standard values ​​other than those mentioned above may be used as appropriate for correction calculations when calculating the weight.

[0109] The area information D22 and weight information D23 serve as indicators for determining the suction priority when multiple foreign objects Cx are detected simultaneously in the suction control unit 62. For example, if multiple foreign objects Cx with approximately the same area information D22 are present, the foreign object Cx with a larger weight information D23 will have a greater adverse effect of increasing the foreign object concentration in valuable resources (glass material GI) if it remains. For this reason, it is preferable to preferentially suction the foreign object Cx with a larger weight information D23, and the weight information D23 serves as an indicator for determining the suction priority, which will be described later.

[0110] The secondary identification information D2 (diagonal length D21, area information D22, weight information D23, etc.) regarding the foreign substance Cx calculated by the parameter calculation unit 611 is sent to the suction control unit 62 together with the original identification information D1.

[0111] <4-3. About the suction control unit> The suction control unit 62 performs suction control processing to control the suction operation of each suction nozzle Nx, nx by controlling the operation of the solenoid valve 7 based on the original identification information D1 and secondary identification information D2 input from the detection unit 61.

[0112] Prior to the execution of the suction control process by the suction control unit 62, various setting registration values ​​are registered in the memory unit 63. Fig. 6 shows examples of the setting registration values ​​stored in the memory unit 63. As shown in Fig. 6, the setting registration values ​​can be broadly categorized as follows: foreign matter characteristic values ​​(A), threshold values ​​(B), suction priority determination conditions (C), production target values ​​(D), and compressor / conveyor control values ​​(E).

[0113] Specifically, the foreign object characteristic values ​​(A) include a density standard value and a height standard value for each foreign object type. In the example of FIG. 6, plastic, washer, and adhesive are registered as foreign object types. The threshold values ​​(B) include a size upper limit value that is the upper limit of the diagonal length D21, a size lower limit value that is the lower limit of the diagonal length D21, a reliability lower limit value that is the lower limit of the reliability, and nozzle diameter allocation values ​​for the first suction nozzle Nx and the second suction nozzle nx. In addition, a suction priority index, which will be described later, is set in the suction priority determination condition (C). In the example of FIG. 6, weight information D23 is set as the suction priority index.

[0114] The production target values ​​(D) include the target yield rate and target purity of the sorting system 1. The compressor / conveyor control values ​​(E) include the compressor capacity value, the maximum number of suction nozzles Nx and nx that can be operated simultaneously, the standard suction cycle time, and the conveyor speed. The maximum number of suction nozzles Nx and nx that can be operated simultaneously may be automatically calculated from the compressor capacity value.

[0115] The standard suction cycle time is the standard period (standard time length) during which the suction nozzles Nx, nx perform suction on one foreign object Cx. The standard suction cycle time is set to be equal to or longer than the minimum time required for an expected standard foreign object Cx to travel from the suction nozzles Nx, nx through the hose 27 to the chamber 28.

[0116] The suction control unit 62 selects the suction nozzles Nx, nx to be operated for sucking the foreign matter Cx based on the identification information D1, D2 input from the detection unit 61. In this case, the suction control unit 62 basically selects the suction nozzles Nx, nx to be operated that have the minimum diameter necessary for sucking the foreign matter Cx based on the area information D11.

[0117] Furthermore, in the case where the suction nozzles Nx, nx are arranged at multiple positions in the width direction for each diameter, as in this embodiment, the suction control unit 62 selects the suction nozzle Nx, nx that has the minimum required diameter and is arranged at a position corresponding to the position information D12 as the nozzle to be operated, based on the area information D11 and the position information D12.

[0118] When detecting a foreign object Cx, the suction control unit 62 refrains from operating the suction nozzles Nx, nx on the foreign object Cx on the condition that the size indicated by the area information D11 is below a predetermined size lower limit.Furthermore, when detecting a foreign object Cx, the suction control unit 62 refrains from operating the suction nozzles Nx, nx on the foreign object Cx on the condition that the size indicated by the area information D11 is above a predetermined size upper limit.

[0119] When a foreign substance Cx is detected, the suction control unit 62 refrains from operating the suction nozzles Nx, nx on the foreign substance Cx on the condition that the reliability information D14 falls below a predetermined lower limit of reliability.

[0120] The suction control unit 62 also calculates the suction time period required to suck the foreign object Cx based on the identification information D1 and D2. The suction control unit 62 calculates the time required for the foreign object Cx to reach the suction area Vx, vx of the suction nozzle Nx, nx to be sucked from the conveying speed of the conveyor 2, and determines the suction time period by referring to the standard suction cycle time. The suction time period begins, for example, when the upstream end of the foreign object Cx reaches the suction area Vx, vx of the selected suction nozzle Nx, nx, and ends after the standard suction cycle time has elapsed. The start of the suction time period may also be when the center position of the foreign object Cx reaches the suction area Vx, vx of the selected suction nozzle Nx, nx.

[0121] When the suction time periods of multiple foreign bodies Cx overlap and an attempt is made to suck up all of the foreign bodies Cx without leaving any behind, the suction control unit 62 determines that a conflict state exists in which it is impossible to suck up all of the foreign bodies Cx without leaving any behind, provided that the number of suction nozzles Nx, nx to be operated exceeds the maximum number that can be operated simultaneously.

[0122] In a conflicting state, the suction control unit 62 determines the suction priority of the multiple foreign objects Cx determined to be in a conflicting state based on the identification information D1 and D2.The suction control unit 62 then limits the foreign objects Cx to be sucked based on the suction priority so that the number of suction nozzles Nx and nx to be operated falls within the maximum number that can be simultaneously operated.The suction priority is determined based on, for example, weight information D23 and type information D13 of the foreign objects Cx.

[0123] In a conflict state, the suction control unit 62 may perform exceptional control, allowing a suction nozzle with a diameter larger than the minimum required diameter to be selected as the suction nozzle to be operated. In this case, the suction control unit 62 first determines the conflict state on the premise that only the suction nozzles Nx, nx with the minimum required diameter are to be operated for each of the multiple foreign objects Cx to be suctioned. Thereafter, if multiple foreign objects Cx with different minimum required diameters are in a conflict state, the suction control unit 62 performs exceptional control on the condition that the suction nozzle Nx, nx with the relatively larger diameter can suck up all of the multiple foreign objects Cx without leaving any behind, based on the size and position information D12 in the area information D11.

[0124] <5. How to create a trained model> A method for creating the trained model M will be described below with reference to Fig. 7. Fig. 7 is a flowchart showing the flow of the trained model M creation process.

[0125] Hereinafter, the selection object Ob used to create the trained model M will be referred to as the "sample material MOb," the image data of the sample material MOb will be referred to as the "teacher image data," and the trainer image data linked to the correct label information will be referred to as the "training dataset."

[0126] First, the sample material MOb is placed on the conveyor 2 (step S101). The sample material MOb is adjusted so that the proportions and sizes of valuable materials and foreign matter contained therein are equivalent to those of the sorting target object Ob that is the actual target of detection and suction removal in the sorting system 1. The sample material MOb contains a mixture of glass material Gl, which is a colorless and transparent valuable material, and multiple opaque foreign matter Cx of different sizes.

[0127] The multiple foreign bodies Cx are objects other than at least one type of glass material Gl, such as pieces of resin (plastic), washers, adhesives, non-magnetic metal pieces, etc. These foreign bodies Cx are generally opaque and / or colored.

[0128] Next, in a shielded environment where ambient light is blocked, the sample material MOb is irradiated with light from the internal light source, the LED lighting 11, while the camera 4, which is the imaging means, captures an image of the sample material MOb (step S102). As a result, teacher image data of the sample material MOb is obtained.

[0129] In step S102, the light emitted from the LED light source 11 is subjected to a first polarization by the first polarizer 16, and the light incident on the camera 4 is subjected to a second polarization by the second polarizer 44. This suppresses reflections of the LED light source 11, and enables the acquisition of good imaging data that reflects the inherent color of the sample material MOb.

[0130] In step S102, it is advisable to acquire a suitable number of training image data in order to achieve a desired level of discrimination accuracy through machine learning.

[0131] Next, a learning data set is created in which type information of the foreign substance Cx included in the training image data is linked to the training image data as a correct answer label (steps S103 to S104).

[0132] Specifically, first, the foreign matter Cx is trimmed from the teacher image data to create a small image (step S103). In this embodiment, the foreign matter Cx is trimmed (cut out) using a rectangular bounding box. At this time, the bounding box is trimmed so as to circumscribe the foreign matter Cx as closely as possible. The trimming process in step S103 may be performed entirely manually by the user, or may be performed partially manually by the user using mechanical assistance such as dedicated software, or may be performed solely by mechanical means.

[0133] Next, each of the small images obtained in the trimming process is linked to a correct label of the type information D13 (e.g., plastic) of the foreign matter Cx, thereby labeling the foreign matter type (step S104). This creates a learning dataset.

[0134] When the foreign object Cx is trimmed, the glass material Gl may also appear in the trimmed area. In this case, the glass material Gl is considered to be a background area, and as a rule, the type information D13 is not labeled. In this case, the teacher image data is captured and trimmed in a state where the glass material Gl and the foreign object Cx are mixed and placed on the conveying surface 22 of the conveyor 2 of the same color as during detection. This allows learning to be performed under conditions similar to those during actual detection. As a result, detection accuracy can be improved.

[0135] In fact, when the inventors of the present application created and performed training on training image data with only foreign objects Cx placed on the conveying surface 22 of the conveyor 2, sufficient detection accuracy was not achieved. Furthermore, when training image data was created with glass material Gl randomly placed on the conveying surface 22 of the conveyor 2 along with foreign objects Cx, and type information D13 was labeled on the glass material Gl in addition to the foreign objects Cx for training, the data processing load became excessive, and detection accuracy worsened. On the other hand, when training image data was created with glass material Gl randomly placed on the conveying surface 22 of the conveyor 2 along with foreign objects Cx, and type information D13 was labeled on only the foreign objects Cx for training, the data processing load was reduced and detection accuracy was good.

[0136] If the trimming process in step S103 is performed solely by mechanical means, there is a possibility that not only the foreign matter Cx but also the valuable glass material Gl will be trimmed. In this case, in the labeling process in step S104, the small image may be deleted from the training dataset, or the small image may be labeled as "not a foreign matter."

[0137] Then, image processing (expansion) such as rotation and inversion is performed on each small image associated with the correct label (step S105). This enriches the learning dataset with various patterns related to various subtle differences, and is expected to improve the accuracy of discrimination for foreign bodies Cx in various positions and states at the time of shooting. It is also expected that a learning dataset sufficient for discrimination can be created from the limited number of training image data.

[0138] Next, the created training dataset is input to an untrained machine learning model, and supervised machine learning of the untrained machine learning model is performed (step S106). In this embodiment, the supervised machine learning in step S106 is performed by deep learning to generate a trained model M.

[0139] In this embodiment, the object detection algorithm YOLO (You only look once) is used as the machine learning model. However, the machine learning model is not limited to this, and a desired algorithm provided by various machine learning libraries or the like may be adopted as the machine learning model. Furthermore, the machine learning model may be a combination of multiple algorithms, or an existing algorithm may be improved.

[0140] Preferably, the machine learning model should be an object detection algorithm capable of performing deep learning, such as the aforementioned YOLO, R-CNN, SSD (Single Shot MultiBox Detector), or other deep convolutional neural networks. In this case, even if the parameters that should be noted as feature quantities for each foreign object Cx are not known in advance, these parameters can be identified by the machine learning model and reflected as feature quantities in the trained model. This allows for accurate detection of foreign objects Cx from the sorting target object Ob, even if the foreign objects Cx have undergone a crushing process and have varied in size and shape (become irregular).

[0141] More preferably, object detection algorithms such as YOLO, R-CNN, and SSD are suitable as machine learning models. These are thought to improve detection accuracy by simultaneously learning the background area of ​​the foreign object (for example, valuables and other foreign objects Cx present behind the foreign object, and the conveying surface 22 of the conveyor 2 having a unique color), and therefore can more effectively suppress false recognition when the valuables in the background are transparent (for example, in the case of glass material GI).

[0142] In particular, by adopting an object detection algorithm (one-stage object detection algorithm) that can simultaneously estimate the object classification and the bounding box trimming range, such as YOLO or SSD, it is possible to increase the processing speed and enable efficient detection. This makes it even more suitable for system control, such as the sorting system 1, which needs to perform processing such as setting a suction task for foreign objects Cx being transported within a relatively short predetermined response time (requiring quick response).

[0143] Furthermore, if an algorithm is used that excludes background regions where foreign objects Cx are unlikely to exist in the early stages of object detection, then when the foreign object Cx is very small, it will be included in the background region and will not be included in the bounding box (it will not be a candidate for the suction task). For example, some versions of YOLO may fall into this category, and such algorithms are thought to be particularly suitable when prioritizing the reduction of yield loss or processing speed.

[0144] On the other hand, if the tolerance for remaining foreign particles Cx is low, an algorithm that targets even very small foreign particles Cx as bounding box partitions (candidates for the suction task) is more suitable. For example, methods derived from SSD or R-CNN may fall into this category. Adopting such an algorithm is advisable when you want to increase the purity of valuable materials by suctioning out as many foreign particles Cx as possible, while tolerating a slight decrease in yield. This is considered to be particularly effective when there is extreme variation in the size of foreign particles Cx.

[0145] Furthermore, when the tolerance for remaining foreign matter Cx is low, it is advisable to improve the detection accuracy of small foreign matter Cx by using different machine learning models, increasing the resolution of camera 4, taking close-up photographs of foreign matter Cx, or other appropriate measures.

[0146] When the machine learning process in step S106 is completed, it is determined whether to terminate learning (step S107). Specifically, validation is performed using a portion of the training image data to confirm the estimation accuracy of the trained model M. The determination in step S107 may be performed mechanically based on the estimation accuracy output from the trained model M, or may be manually confirmed and determined by the user.

[0147] In step S107, if sufficient estimation accuracy is not obtained (step S107: No), the process returns to step S106, where various conditions such as the weighting of various parameters during machine learning are appropriately changed (model tuning), and supervised machine learning is performed again.

[0148] If sufficient estimation accuracy is not obtained (step S107: No), the training data set may be expanded as necessary before re-learning in step S106. For example, new training image data may be obtained by photographing the same sample material MOb in a different position, or by photographing a different sample material MOb. Furthermore, the training data set may be expanded by performing more image processing patterns on the same training image data.

[0149] On the other hand, if sufficient estimation accuracy is obtained in step S107 (step S107: Yes), various parameters of the trained model M are determined, and the creation of the trained model M by machine learning is completed (step S108). The trained model M that has been created is incorporated into the detection unit 61. This completes all steps of the trained model M creation process.

[0150] If the trained model M created in this way has sufficient estimation accuracy, when the presence of a foreign object Cx in a sorting object Ob being transported on the conveyor 2 is detected, the rectangular bounding box indicated by the area information D11 of the foreign object Cx will approximately circumscribe the area in which the foreign object Cx actually exists.

[0151] That is, the area included in the area information D11 is accurately distinguished from the background area (including the glass material Gl) and is represented by a bounding box defined by a rectangle (rectangle or square) circumscribing the foreign object Cx.

[0152] If the trimming is inaccurate, the bounding box may not circumscribe the foreign object Cx, and may be larger or smaller than the actual size of the foreign object Cx. For this reason, it is desirable to trim the teacher image data accurately in step S103.

[0153] The above machine learning may be performed on the information processing device 6 of the selection system 1, or may be performed on another computer terminal (PC) equipped with an appropriate GPU or the like, i.e., a dedicated terminal for machine learning. In this case, the information processing device 6 and the dedicated terminal may be connected via a wired or wireless network, and a training dataset may be transferred from the information processing device 6 to the dedicated terminal, or the created trained model M may be sent from the dedicated terminal to the information processing device 6.

[0154] In this way, computer terminals on a network, such as servers and dedicated terminals, including both physical and cloud computing, may substitute for or complement part or all of the functions of the information processing device 6 of the sorting system 1. In this case, these computer terminals, as information processing devices 6, fall within the technical scope of the present invention.

[0155] <6. Suction control processing> The suction control process in the sorting system 1 of this embodiment will be described below in order.

[0156] <6-1. Initial Setup Procedure> Prior to the suction control process in the sorting system 1, various setting registration values ​​are input to the sorting system 1 as initial settings.

[0157] 6 are input to the storage unit 63 of the information processing device 6. Specifically, the storage unit 63 of the information processing device 6 inputs the density standard value and height standard value for each type of foreign matter, the upper size limit value, the lower size limit value, the lower reliability limit value, the nozzle diameter allocation values ​​for the first suction nozzle Nx and the second suction nozzle nx, the suction priority index (determination condition), and the target yield rate and target purity for the sorting system 1.

[0158] The compressor / conveyor control panel 20 is also started, and each set registered value of the compressor / conveyor control value (E) is input to the compressor / conveyor control panel 20. Specifically, the compressor capacity value, the maximum number of suction nozzles Nx and nx that can be simultaneously operated, the standard suction cycle time, and the conveyor speed are input to the compressor / conveyor control panel 20. Note that the maximum number of suction nozzles Nx and nx that can be simultaneously operated may be automatically calculated from the compressor capacity value. Each set registered value of the compressor / conveyor control value (E) input to the compressor / conveyor control panel 20 may be transmitted to the information processing device 6 and stored in the memory unit 63.

[0159] The density standard value may be a value obtained by referring to a data sheet or the like for each material constituting the various foreign matter Cx, or may be a value obtained from the actual measurement value of the density of the various foreign matter Cx. The height standard value may be set, for example, based on the measurement results of an actual measurement of the sorting object Ob. Each of these standard values ​​may be a value represented by various indices such as the average or median of multiple measurement values.

[0160] The upper size limit is the upper limit of the size (diagonal length D21) of the foreign object Cx that can be sucked by the first suction nozzle Nx, which has the largest diameter among the suction nozzles Nx and nx. Foreign objects Cx that exceed the upper size limit cannot be removed by suction in the suction unit 5, so even if the suction nozzles Nx and nx are operated, the foreign object Cx cannot be removed and only the surrounding valuables are sucked in. For this reason, an upper size limit is set.

[0161] The size lower limit is the lower limit of the size (diagonal length D21) of the foreign matter Cx to be sucked. If the foreign matter Cx is too small, removing the foreign matter Cx will only result in a slight increase in purity, while even using the second suction nozzle nx with the smallest diameter will result in a decrease in yield due to the suction of surrounding valuable materials. For this reason, a size lower limit is set to prevent excessive suction.

[0162] The reliability lower limit is the lower limit of the reliability information D14 of the foreign object Cx to be sucked. If the reliability information D14 of the foreign object Cx is too small, there is a high risk that the detection result of the foreign object Cx will be a false positive. In other words, it is uncertain whether the foreign object Cx is a foreign object Cx, and there is a high possibility that the foreign object is actually glass material Gl. For this reason, the reliability lower limit is set from the perspective of preventing excessive suction of glass material Gl, which is a valuable resource.

[0163] Furthermore, the reliability information D14 in this embodiment may be a value indicating the accuracy of the area information D11 or the position information (coordinates) D12. In this case, if the reliability information D14 is too small, it is considered that it is uncertain whether or not a foreign object Cx exists within the bounding box. In such a case, removing the foreign object Cx may result in unnecessary suction of surrounding glass material Gl. For this reason, setting a reliability lower limit value is also considered effective from the viewpoint of preventing excessive suction.

[0164] The nozzle diameter allocation values ​​for the first suction nozzle Nx and the second suction nozzle nx are thresholds for determining which suction nozzle Nx, nx to operate based on the size (diagonal length D21) of the foreign object Cx. In this embodiment, the suction nozzles Nx, nx have two different diameters: the first suction nozzle Nx and the second suction nozzle nx. Therefore, there is only one threshold for the nozzle diameter allocation value, which matches the inner diameter of the second suction nozzle nx. If multiple different diameters of suction nozzles are provided, it is advisable to determine the nozzle diameter allocation value according to the inner diameter of each suction nozzle.

[0165] The suction priority index is set appropriately depending on the type of sorting object Ob and the use of the valuables recovered after the sorting process. In this embodiment, weight information D23 is set as a reference for the suction priority index. As a result, when multiple foreign objects Cx are simultaneously targeted for suction, the heavier foreign objects Cx are preferentially sucked and removed, thereby improving the purity of the valuables in the recovered materials. Furthermore, if there is a foreign object type that would be problematic if mixed into the recovered valuables due to the use of the recovered valuables, that foreign object type may be set and registered as a suction priority index.

[0166] The target yield rate of the sorting and recovery process and the target purity of the valuable material (glass material Gl) are set and registered as necessary.

[0167] These set and registered values ​​are determined according to production target values ​​(D) such as the property difference of the sorting object Ob to be recycled, the target purity, and the target yield rate.

[0168] For example, in the case of the present embodiment where glass material Gl is a valuable resource, if the foreign matter Cx is minute plastic or the like, it may have little effect on purity by volatilizing in the melting and refining process after recovery in the sorting system 1. For this reason, it is preferable to initially set appropriate set registered values ​​in consideration of the trade-off between yield and purity so that the yield does not drop significantly by pursuing a purity that is excessive compared to the requirement.

[0169] In contrast to the above, when higher purity of the glass material Gl is required or when the valuables contained in the sorting object Ob have a low tolerance for remaining foreign matter Cx, appropriate set registered values ​​may be initially set to maximize the purity of the valuables while allowing for a slight decrease in yield. For example, the frequency of decisions to postpone may be reduced by relatively lowering each lower limit value for operation suppression (such as the size lower limit value and the reliability lower limit value).

[0170] <6-2. Sorting and processing of valuables> The flow of the valuable resource sorting process in the sorting system 1 of this embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the flow of the valuable resource sorting process in the sorting system 1.

[0171] In the valuable resource sorting process, when the sorting system 1 is started up, first, the information processing device 6 and the compressor / conveyor control panel 20 read various setting registered values ​​(step S201).

[0172] Then, the operation of the sorting system 1 is started (step S202). When the operation of the sorting system 1 is started, first, the conveyor 2 and the compressor 25 are started. Next, when the LED lights 11 are turned on and the camera 4 is ready to take an image, the objects to be sorted Ob start to be fed from the feeding chute 3 to the conveyor 2. At the same time, the camera 4 starts taking images periodically and detecting foreign objects Cx from the image data (step S203).

[0173] The objects Ob to be sorted, which are placed near the upstream end of the conveyor 2, are leveled by a scraper 12 provided near the entrance of the hood 9, and proceed into the imaging range 10 inside the hood 9 while minimizing overlapping. Then, in a shielded environment where ambient light is blocked, the camera 4 captures the objects Ob while irradiating them with light from the LED light 11. This acquires imaging data. Note that, at this time, as in the creation of the trained model M, the light emitted from the LED light 11 is subjected to a first polarization by the first polarizer 16, and the light entering the camera 4 is subjected to a second polarization by the second polarizer 44.

[0174] The camera 4 periodically captures images at a predetermined frame rate (for example, about 75 times per second) according to its performance, and sequentially transfers the captured image data to the information processing device 6. The information processing device 6 sequentially inputs the received captured image data to the trained model M of the detection unit 61. As a result, the trained model M detects foreign objects Cx in the input captured image data.

[0175] At this time, the camera 4 does not have to transfer all of the acquired imaging data to the information processing device 6. In this embodiment, the camera 4 selectively transfers imaging data to the information processing device 6 at predetermined time intervals according to the speed of the conveyor 2 so as to minimize overlap and omission of areas on the transport surface 22 of the conveyor 2. This allows the information processing device 6 to acquire continuous imaging data in chronological order without excess or deficiency.

[0176] After starting photography and foreign object detection (step S203), the information processing device 6 monitors whether the trained model M has detected a foreign object Cx (step S204). If it determines that a foreign object Cx has not been detected (step S204: No), the information processing device 6 proceeds to step S206 and determines whether an operation termination command has been input (step S206). If an operation termination command has not been input (step S206: No), the information processing device 6 returns to step S204 and continues monitoring whether a foreign object Cx has been detected. On the other hand, if an operation termination command has been input (step S206: Yes), the valuable resource sorting process is terminated.

[0177] In step S204, if it is determined that the trained model M has detected a foreign object Cx (step S204: Yes), the information processing device 6 starts the suction control process (step S205) described below for the detected foreign object Cx. Thereafter, in parallel with the suction control process (S205) for the foreign object Cx, the process proceeds to step S206. If an operation end command has not been input (step S206: No), the process returns to step S204 and monitors whether the trained model M has detected the next foreign object Cx. On the other hand, if an operation end command has been input (step S206: Yes), the valuables sorting process is terminated.

[0178] <6-3. Suction control process> Next, the flow of the suction control process (step S205) for the foreign matter Cx detected by the trained model M will be described with reference to Figures 9 to 13. Figure 9 is a flowchart showing the flow of the suction control process in the sorting system 1. Figures 10 to 13 are diagrams showing an example of a foreign matter Cx and suction nozzles Nx, nx arranged at positions corresponding to the foreign matter Cx.

[0179] As described above, when the trained model M detects a foreign object Cx in the valuable resource sorting process, the information processing device 6 performs a suction control process (step S205) for the foreign object Cx. As shown in Fig. 9, in the suction control process, the suction control unit 62 first acquires original identification information D1 output from the trained model M together with the detection of the foreign object Cx (step S301). The original identification information D1 includes area information D11, position information (coordinates) D12, type information D13, and reliability information D14.

[0180] Then, the parameter calculation unit 611 calculates secondary identification information D2 from the original identification information D1 (step S302). The secondary identification information D2 includes a diagonal length D21, area information D22, and weight information D23. The suction control unit 62 controls the operation of the suction nozzles Nx and nx based on the original identification information D1 and the secondary identification information D2 in the following steps.

[0181] The suction control unit 62 determines whether the diagonal length D21 (size of the area information D11) of the detected foreign object Cx is within a predetermined range (step S303). Specifically, in step S303, the suction control unit 62 determines whether the diagonal length D21 of the detected foreign object Cx is equal to or greater than a predetermined size lower limit and equal to or less than a predetermined size upper limit. If it determines that the diagonal length D21 of the detected foreign object Cx is not within the predetermined range (step S303: No), the suction control unit 62 refrains from operating the suction nozzles Nx and nx on the foreign object Cx and terminates the suction control process for the foreign object Cx. If the diagonal length D21 of the detected foreign object Cx is not within the predetermined range, the diagonal length D21 may be below the size lower limit or may be above the size upper limit.

[0182] Figure 10 shows an example in which the diagonal length D21 of foreign object Cx exceeds the upper size limit. In Figure 10, the bounding box BBa of the detected foreign object Ca is indicated by a dashed line. In the example of Figure 10, the diagonal length D21 of the bounding box BBa is greater than the upper size limit, which is equal to the inner diameter of the first suction nozzle Nx (N1, N2) with the largest opening. Therefore, in step S303, the suction control unit 62 determines that the diagonal length D21 of the detected foreign object Ca is not within the predetermined range, and operation of the suction nozzles Nx, nx with respect to the foreign object Ca is postponed.

[0183] FIG. 11 shows an example in which the diagonal length D21 of a foreign object Cx is below the lower size limit. In FIG. 11, the bounding box BBb of the detected foreign object Cb is indicated by a dashed line. In the example of FIG. 11, the detected foreign object Cb is very small and has little impact on the purity of the valuable material in the recovered material. The diagonal length D21 of the bounding box BBb of the foreign object Cb is smaller than the predetermined lower size limit. Therefore, in step S303, the suction control unit 62 determines that the diagonal length D21 of the detected foreign object Cb is not within the predetermined range, and operation of the suction nozzles Nx and nx on the foreign object Cb is postponed.

[0184] If it is determined in step S303 that the diagonal length D21 of the detected foreign object Cx is within a predetermined range (step S303: Yes), the suction control unit 62 then determines whether the reliability information D14 of the detected foreign object Cx is greater than a predetermined lower limit of reliability (step S304). If the reliability information D14 of the detected foreign object Cx is below the lower limit of reliability (step S304: No), the suction control unit 62 refrains from operating the suction nozzles Nx, nx on the foreign object Cx and terminates the suction control process for the foreign object Cx.

[0185] FIG. 12 shows an example in which the reliability information D14 for a foreign object Cx falls below the lower limit of reliability. In FIG. 12, the detected foreign object Cc is not actually a foreign object, but an overlapping piece of glass material Gl. However, due to the overlapping, the foreign object Cc is erroneously detected as a foreign object such as plastic. In this case, the reliability information D14 for the foreign object Cc output by the trained model M falls below the lower limit of reliability. For example, the lower limit of reliability is 0.60 (60%), while the reliability information D14 for the foreign object Cc is 0.42 (42%). Therefore, in step S304, the suction control unit 62 determines that the reliability information D14 for the detected foreign object Cc falls below the lower limit of reliability, and the operation of the suction nozzles Nx and nx for the foreign object Cc is postponed.

[0186] Note that such false detection due to overlapping materials is more pronounced when the valuables are colorless and transparent. Therefore, when the valuables are substantially colorless and transparent materials such as glass material Gl and the foreign matter Cx is opaque and / or colored, the advantage of not suctioning based on the reliability information D14 is significant, and it is even more effective in preventing excessive suction.

[0187] In step S304, if it is determined that the reliability information D14 of the detected foreign matter Cx is equal to or greater than the lower limit reliability value (step S304: Yes), the suction control unit 62 proceeds to step S305. Note that in this suction control process, the order of the decision to postpone based on the foreign matter size in step S303 and the decision to postpone based on the reliability in step S304 may be reversed.

[0188] In step S305, the suction control unit 62 determines the suction nozzles Nx, nx to be used to suck the detected foreign matter Cx, and calculates the suction time period for the suction nozzles Nx, nx (step S305).

[0189] In step S305, the suction control unit 62 first selects one of the suction nozzles Nx, nx having the minimum diameter required to suck the detected foreign matter Cx as an operation target based on the area information D11. Then, the suction control unit 62 calculates the suction time period required to suck the detected foreign matter Cx based on the identification information D1, D2.

[0190] Specifically, the suction control unit 62 first compares the diagonal length D21 calculated from the area information D11 with the nozzle diameter allocation value to determine whether the suction nozzle Nx, nx with the minimum diameter required to suck the detected foreign matter Cx is the first suction nozzle Nx or the second suction nozzle nx. If the diagonal length D21 falls within the range of the nozzle diameter allocation value for the first suction nozzle Nx, the suction control unit 62 selects the first suction nozzle Nx as the nozzle to be operated. If the diagonal length D21 falls within the range of the nozzle diameter allocation value for the second suction nozzle nx, the suction control unit 62 selects the second suction nozzle nx as the nozzle to be operated.

[0191] Then, the suction control unit 62 selects one suction nozzle Nx, nx from the multiple first suction nozzles Nx or multiple second suction nozzles nx based on the position information D12 of the detected foreign object Cx, which corresponds to the suction lane Lx, lx to which the detected foreign object Cx belongs.

[0192] Next, the suction control unit 62 calculates a suction time period during which the suction nozzle Nx, nx selected to suck the detected foreign object Cx will be operated. The suction control unit 62 calculates the time required for the foreign object Cx to reach the suction area Vx, vx of the suction nozzle Nx, nx to be sucked from the conveyor 2 transport speed, and determines the suction time period by referring to the standard suction cycle time. The suction time period begins, for example, when the upstream end of the foreign object Cx reaches the suction area Vx, vx of the selected suction nozzle Nx, nx and ends after the standard suction cycle time has elapsed. The start of the suction time period may also be when the center position of the foreign object Cx reaches the suction area Vx, vx of the selected suction nozzle Nx, nx.

[0193] In this way, in step S305, one suction nozzle Nx, nx that will suck the detected foreign matter Cx and the suction time period during which the suction nozzle Nx, nx operates are determined. Hereinafter, the suction nozzle Nx, nx and the suction time period determined in step S305 are referred to as the "suction task" of the foreign matter Cx.

[0194] 13 and 14 show examples of a detected foreign object Cx and the suction nozzles Nx and nx selected in step S305. In the example of Fig. 13, the diagonal length D21 of the detected foreign object Cd, i.e., the diagonal length of the bounding box BBd, falls within the range of the nozzle diameter allocation value for the first suction nozzle Nx. Therefore, the suction control unit 62 determines to use one of the first suction nozzles Nx (N1, N2) to suction the foreign object Cd.

[0195] Next, the suction control unit 62 determines which suction lane Lx of the first suction nozzle Nx the position information D12 of the detected foreign object Cd, i.e., the widthwise position of the center point Pd of the bounding box BBd, overlaps with. In the example of Figure 13, the widthwise position of the center point Pd of the bounding box BBd is on the suction lane L1 corresponding to the first suction nozzle N1, indicated by the two-dot chain line. Therefore, the suction control unit 62 determines to use the first suction nozzle N1 to suction the foreign object Cd.

[0196] 14, the diagonal length D21 of the detected foreign matter Ce, i.e., the diagonal length of the bounding box BBe, falls within the range of the nozzle diameter allocation value of the second suction nozzle nx. Therefore, the suction control unit 62 determines to use one of the second suction nozzles nx (n1 to n4) to suck the foreign matter Ce.

[0197] Next, the suction control unit 62 determines which suction lane lx of the second suction nozzle nx the position information D12 of the detected foreign object Ce, i.e., the widthwise position of the center point Pe of the bounding box BBe, overlaps with. In the example of Figure 14, the widthwise position of the center point Pe of the bounding box BBe is on the suction lane l3 corresponding to the second suction nozzle n3, indicated by the two-dot chain line. Therefore, the suction control unit 62 determines to use the second suction nozzle n3 to suction the foreign object Ce.

[0198] Returning to Figure 9, after determining the suction nozzles Nx and nx in step S305, the suction control unit 62 determines whether the detected foreign object Cx is in a conflict with other foreign objects Cx (step S306). A conflict occurs when the suction time periods of multiple foreign objects Cx overlap, making it impossible to completely suck up all of the foreign objects Cx, provided that attempting to completely suck up all of the foreign objects Cx would exceed the maximum number of suction nozzles Nx and nx that can be simultaneously operated.

[0199] In determining whether a conflict exists in step S306, if a number of foreign objects Cx exceeding the maximum number that can be simultaneously operated is detected, it is determined whether the time periods of the suction tasks for the detected multiple foreign objects Cx overlap. If the number of suction tasks with overlapping time periods exceeds the maximum number of suction nozzles Nx, nx that can be simultaneously operated, a conflict is determined.

[0200] Here, a specific example of determining a conflict state will be described with reference to FIGS. 15 and 16. FIG. 15 shows an example in which five foreign objects Cf to Cj are detected simultaneously but there is no conflict state. FIG. 15(a) shows an example of five foreign objects Cf to Cj and suction nozzles N1 to N3 and n1 to n6 arranged at positions corresponding to these foreign objects Cf to Cj. FIG. 15(b) shows suction tasks for the five foreign objects Cf to Cj. In FIG. 15(b), each suction task is indicated by a thick line arranged vertically. On the left side of each suction task, which suction nozzle Nx or nx is used for which foreign object Cx is written in the format Cx-Nx or Cx-nx. The horizontal axis represents time, and the horizontal length of the thick line represents the length of the suction period.

[0201] Foreign matter Cf is not subjected to suction in step S303 because its diagonal length D21 is smaller than the lower limit of size. Foreign matter Cg is an overlapping portion of glass material Gl, and its reliability information D14 is smaller than the lower limit of reliability, so it is not subjected to suction in step S304. Therefore, as shown in FIG. 15(b), a suction task is not set for foreign matter Cf and foreign matter Cg in step S305, and they are not included in the determination of the conflict state in step S306.

[0202] Therefore, in step S306, it is determined whether or not a conflict exists for only three foreign objects Ch, Ci, and Cj. First, the number of foreign objects Ch, Ci, and Cj for which a conflict is to be determined is three, which exceeds the maximum number of foreign objects that can be simultaneously operated, which is "2." Therefore, it is next determined whether or not the time periods of the suction tasks overlap.

[0203] In step S305, the suction nozzles Nx, nx to be activated are selected as follows: first suction nozzle N1 for foreign object Ch, first suction nozzle N2 for foreign object Ci, and first suction nozzle N3 for foreign object Cj. Also in step S305, the suction time period for each foreign object Ch, Ci, Cj is calculated. The start time of the suction time period for each foreign object Ch, Ci, Cj is the time when the upstream end of the foreign object Ch, Ci, Cj reaches the suction range V1, V2, V3 of the first suction nozzle N1, N2, N3, which is the suction target nozzle. The length of the suction time period for each foreign object Ch, Ci, Cj is the standard suction cycle time.

[0204] The suction tasks for each of the foreign bodies Ch, Ci, and Cj determined in this manner are shown in Figure 15(b). As shown in Figure 15(b), the suction time period for foreign body Ch overlaps for the most part with the suction time period for foreign body Ci. However, the suction time period for foreign body Cj does not overlap with the suction time periods for either foreign body Ch or Ci. Therefore, there is no time when three or more different suction nozzles Nx are operating simultaneously. In other words, the number of suction tasks with overlapping time periods is "2," which does not exceed the maximum number of suction nozzles Nx and nx that can be simultaneously operated, "2," so in step S306, it is determined that there is no conflict.

[0205] FIG. 16 shows an example of a case where four detected foreign objects Ck to Cn are in a conflicting state. FIG. 16(a) shows an example of four foreign objects Ck to Cn and suction nozzles N1 to N3 and n1 to n6 arranged at positions corresponding to these foreign objects Ck to Cn. FIG. 16(b) shows suction tasks for the four foreign objects Ck to Cn. FIG. 16(c) will be described later. Note that in FIG. 16(b) and subsequent figures showing suction tasks, areas where there are three or more overlapping suction tasks are surrounded by dashed lines, and areas where the maximum number of overlapping suction tasks is four or more are shown in gray.

[0206] 16, in step S305, the suction nozzles Nx, nx to be operated are selected as follows: first suction nozzle N1 for foreign matter Ck, first suction nozzle N2 for foreign matter Cl, first suction nozzle N3 for foreign matter Cm, and second suction nozzle n1 for foreign matter Cn. Also in step S305, the suction time periods for each of the foreign matters Ck to Cn are calculated.

[0207] The suction tasks for each of the foreign bodies Ck-Cn determined in this manner are shown in Figure 16(b). As shown in Figure 16(b), there is a time period in which all four suction tasks for the four foreign bodies Ck-Cn overlap. Also, before and after this time period, there are time periods in which three suction tasks overlap. In other words, the number of overlapping suction tasks in the time period is "4," which exceeds the maximum number of suction nozzles Nx and nx that can be simultaneously operated, which is "2," so a conflict is determined in step S306.

[0208] Returning to Fig. 9, if it is determined in step S306 that a conflict state exists (step S306: Yes), the suction control unit 62 proceeds to a conflict control step in step S307. The conflict control step (step S307) will be described later.

[0209] On the other hand, if it is determined in step S306 that there is no conflict (step S306: No), the suction control unit 62 confirms the suction task calculated in step S305 and hands over the suction task to the PLC 8 (step S308). In accordance with the suction task handed over from the suction control unit 62, the PLC 8 performs a suction operation by opening the solenoid valve 7 corresponding to the designated suction nozzle Nx, nx for the designated suction time period (step S309). If it is determined that there is no conflict, the suction control process for the detected foreign object Cx described above ends.

[0210] <6-4. Control process during conflict (no exceptions to nozzle diameter selection)> Next, the conflict control process (step S307) when it is determined that a conflict state exists in the suction control process will be described. Here, the case where there is no "exceptional selection of nozzle diameter" (described later) will be described with reference to Figs. 16 to 18. Fig. 17 is a flowchart showing the flow of the conflict control process when there is no exception selection of nozzle diameter. Fig. 18 is a diagram showing an example where three detected foreign objects Co to Cq are in a conflict state.

[0211] 17, in the conflict control step (no exceptional selection of nozzle diameter), the suction control unit 62 first determines whether multiple foreign bodies Cx can be successively sucked up using one suction nozzle Nx, nx without changing the suction nozzle Nx, nx selected for each of the competing foreign bodies Cx in step S305 (step S401). That is, it determines whether the same suction nozzle Nx, nx is selected among the multiple competing foreign bodies Cx and whether there are any that have overlapping suction time periods.

[0212] If it is determined in step S401 that multiple foreign bodies Cx can be continuously sucked without changing the suction nozzles Nx, nx for each foreign body Cx (step S401: Yes), the suction control unit 62 integrates the suction tasks of the foreign bodies Cx that are to be continuously sucked using the same suction nozzles Nx, nx (step S402). Specifically, the start of the suction time period of the foreign body Cx whose suction time period starts earliest among the foreign bodies Cx to be continuously sucked is set as the start of the suction time period of the integrated task, and the end of the suction time period of the foreign body Cx whose suction time period ends latest among the foreign bodies Cx to be continuously sucked is set as the end of the suction time period of the integrated task.

[0213] Fig. 18 shows an example of three foreign objects Co to Cq in a conflicting state that can be task integrated. Fig. 18(a) shows an example of three foreign objects Co to Cq and suction nozzles N1 to N3 and n1 to n6 arranged at positions corresponding to these foreign objects Co to Cq. Fig. 18(b) shows suction tasks for the three foreign objects Co to Cq. Fig. 18(c) shows suction tasks for the three foreign objects Co to Cq after task integration.

[0214] In the example of Figure 18, as shown in Figure 18(b), of the three conflicting foreign objects Co to Cq, the suction tasks for two foreign objects Co and Cp have overlapping suction time periods for the same second suction nozzle n3. Therefore, the task for foreign object Co and the task for foreign object Cp can be integrated. Figure 18(c) shows an integrated task (Co / Cp-n3) that integrates the tasks for foreign object Co and Cp, and a task for foreign object Cq (Cq-N3). As a result, the number of overlapping suction tasks becomes "2," which does not exceed the maximum number of suction nozzles Nx and nx that can be simultaneously operated, which is "2," and therefore the conflict is resolved.

[0215] Returning to Fig. 17, after integrating the suction tasks without changing the suction nozzles Nx and nx in step S402, the suction control unit 62 again determines whether or not a conflict state exists (step S403).

[0216] If it is determined in step S403 that there is no conflict, i.e., that the conflict has been resolved (step S403: No), the suction control unit 62 proceeds to step S406, where it determines the suction task after being integrated in step S402, and hands over the suction task to the PLC 8 (step S406). In accordance with the suction task handed over from the suction control unit 62, the PLC 8 performs a suction operation by opening the solenoid valve 7 corresponding to the designated suction nozzle Nx, nx for the designated suction time period (step S406). This completes the suction control process for the detected foreign object Cx.

[0217] If it is determined in step S403 that the conflict state still exists, that is, that the conflict state has not been resolved (step S403: Yes), the process proceeds to steps S404 and S405, where the attraction task is deleted.

[0218] On the other hand, in step S401, if it is determined that multiple foreign bodies Cx cannot be continuously sucked without changing the suction nozzles Nx, nx for each foreign body Cx (step S401: No), the suction control unit 62 proceeds to steps S404 to S405 and deletes the suction task.

[0219] When deleting a suction task, the suction control unit 62 first determines the priority of the multiple suction tasks that are in a conflicting state (step S404). In this embodiment, the suction priority is determined based on the weight information D23. Therefore, the suction control unit 62 compares the total values ​​of the weight information D23 of the foreign matter Cx for each suction task, and determines the priority of the suction task in descending order of the total value.

[0220] Thereafter, the suction control unit 62 selects, in descending order of priority according to the priorities determined in step S404, as suction tasks to be executed a number equal to the maximum number of suction nozzles Nx, nx that can be simultaneously operated. Meanwhile, the suction control unit 62 deletes the suction tasks that were not selected (step S405). As a result, the number of suction tasks that overlap in time period becomes the same as the maximum number that can be simultaneously operated, and the conflict state is resolved.

[0221] Following step S405, the suction control unit 62 determines the remaining suction task and hands it over to the PLC 8 (step S406). In accordance with the suction task handed over from the suction control unit 62, the PLC 8 performs a suction operation by opening the solenoid valve 7 corresponding to the designated suction nozzle Nx, nx for the designated suction time period (step S406). This completes the suction control process for the detected foreign object Cx.

[0222] Here, reference is made again to FIG. 16. As shown in FIG. 16(b), there are time periods in which the suction tasks for all four foreign objects Ck to Cn in FIG. 16 overlap. Therefore, it is necessary to reduce the number of suction tasks in the overlapping time periods to the maximum number of suction nozzles Nx, nx that can be simultaneously operated, which is "2." As shown in FIG. 16(b), none of these foreign objects Ck to Cn share the same suction nozzles Nx, nx as their suction targets. Therefore, no suction tasks are to be integrated in step S401. Therefore, in step S404, the weight information D23 for each of the foreign objects Ck to Cn is compared.

[0223] In the example of Figure 16, the diagonal lengths 21 of the respective foreign objects Ck-Cn are actually measured in the order Cl > Cm > Ck > Cn. Meanwhile, the standard density value of the washer, which is the foreign object type of the foreign objects Ck and Cm, is greater than the standard density value of the plastic, which is the foreign object type of the foreign objects Cl and Cn. As a result, in step S404, the magnitude of the weight information D23 is in the order Cm > Ck > Cl > Cn. Therefore, in step S405, the suction tasks for the foreign objects Cl and Cn are deleted.

[0224] Figure 16(c) shows the suction tasks for the foreign objects Ck to Cn after the suction task deletion. As shown in Figure 16(c), by deleting the suction task in step S405, the number of suction tasks overlapping in the time period becomes "2", which does not exceed the maximum number of suction nozzles Nx and nx that can be operated simultaneously, which is "2", so the conflict state is no longer present.

[0225] <6-5. Control process during conflict (with exception of nozzle diameter)> The conflict control process (step S307) when a conflict is determined to exist in the suction control process will be described with reference to Figs. 19 to 21, with reference to the case where "exceptional selection of nozzle diameter" (exception control) is performed. Fig. 19 is a flowchart showing the flow of the conflict control process when exceptional selection of nozzle diameter (exception control) is performed. Figs. 20 and 21 are diagrams showing an example where exception control is performed on a detected foreign object Cx.

[0226] 19, in the conflict control step (with exceptional selection of nozzle diameter), the suction control unit 62 first determines whether multiple foreign bodies Cx can be successively sucked up using one suction nozzle Nx, nx without changing the suction nozzle Nx, nx selected for each of the competing foreign bodies Cx in step S305 (step S501). That is, it determines whether the same suction nozzle Nx, nx is selected among the competing foreign bodies Cx and whether there are any that have overlapping suction time periods.

[0227] If it is determined in step S501 that multiple foreign bodies Cx can be continuously sucked without changing the suction nozzles Nx, nx for each foreign body Cx (step S501: Yes), the suction control unit 62 integrates the suction tasks of the foreign bodies Cx that are to be continuously sucked using the same suction nozzles Nx, nx (step S502). Specifically, the start of the suction time period of the foreign body Cx whose suction time period starts earliest among the foreign bodies Cx to be continuously sucked is set as the start of the suction time period of the integrated task, and the end of the suction time period of the foreign body Cx whose suction time period ends latest among the foreign bodies Cx to be continuously sucked is set as the end of the suction time period of the integrated task.

[0228] After integrating the suction tasks without changing the suction nozzles Nx and nx in step S502, the suction control unit 62 again determines whether or not a conflict state exists (step S503).

[0229] If it is determined in step S503 that there is no conflict, i.e., that the conflict has been resolved (step S503: No), the suction control unit 62 proceeds to step S509, where it determines the suction task after being integrated in step S502, and hands over the suction task to the PLC 8 (step S509). In accordance with the suction task handed over from the suction control unit 62, the PLC 8 performs a suction operation by opening the solenoid valve 7 corresponding to the designated suction nozzle Nx, nx for the designated suction time period (step S510). This completes the suction control process for the detected foreign object Cx.

[0230] If it is determined in step S503 that the conflict state still exists, that is, that the conflict state has not been resolved (step S503: Yes), the suction control unit 62 proceeds to step S504.

[0231] On the other hand, in step S501, if it is determined that it is not possible to continuously suck multiple foreign substances Cx without changing the suction nozzles Nx and nx for each foreign substance Cx (step S501: No), the suction control unit 62 also proceeds to step S504.

[0232] In step S504, the suction control unit 62 determines whether multiple foreign bodies Cx can be continuously sucked up using a single suction nozzle Nx, nx by changing the diameter of the suction nozzle Nx, nx to be sucked for any of the competing foreign bodies Cx (step S504). Note that "changing the diameter" refers to changing the suction target from a suction nozzle with a small diameter to a suction nozzle with a large diameter. That is, in this embodiment, changing the second suction nozzle nx to the first suction nozzle Nx with a larger diameter is referred to as "changing the diameter." For example, if the suction unit 5 has suction nozzles with three diameters (large, medium, and small), changing from a small nozzle to a medium nozzle or a large nozzle, and changing from a medium nozzle to a large nozzle are referred to as "changing the diameter."

[0233] In step S504, if it is determined that multiple foreign objects Cx can be continuously sucked by changing the aperture of the suction nozzle Nx, nx for any of the foreign objects Cx (step S504: Yes), the suction control unit 62 integrates the suction tasks of the foreign objects Cx that are to be continuously sucked by the same suction nozzle Nx, nx (step S505). Specifically, the suction control unit 62 generates new suction tasks by changing the suction nozzle Nx, nx that is to be operated for some of the foreign objects Cx that are to be continuously sucked, and then integrates the suction tasks.

[0234] FIG. 20 shows an example of the same four foreign bodies Ck to Cn as in FIG. 16. FIG. 20(a) shows an example of the four foreign bodies Ck to Cn and suction nozzles N1 to N3 and n1 to n6 arranged at positions corresponding to these foreign bodies Ck to Cn. FIG. 20(b) shows the suction tasks for the four foreign bodies Ck to Cn. FIG. 20(c) shows the suction tasks for the four foreign bodies Ck to Cn after task integration. In the example of FIG. 20, unlike the example of FIG. 16, the suction tasks for the four foreign bodies Ck to Cn in a conflicting state can be integrated by changing the diameter of the suction nozzles Nx and nx, i.e., by performing exceptional selection (exception control) of the nozzle diameter.

[0235] In Figure 20(b), the suction tasks for the suction nozzles Nx, nx initially determined to be activated in step S305 of the suction control process are indicated by thick solid lines. As shown in Figure 20(b), there are time periods in which four suction tasks overlap: the suction task for foreign object Ck (Ck-N1), the suction task for foreign object Cl (Cl-N2), the suction task for foreign object Cm (Cm-N3), and the suction task for foreign object Cn (Cn-n1). Therefore, the number of overlapping suction tasks in the time period is "4," which exceeds the maximum number of suction nozzles Nx, nx that can be simultaneously operated, which is "2."

[0236] 20(a), the foreign object Cn, which is the target of operation of the small-diameter second suction nozzle n1, is located in a position that belongs to the suction lane of the large-diameter first suction nozzle N1, which is the target of operation of another foreign object Ck. Therefore, by changing the diameter of the target of operation of the foreign object Cn to the first suction nozzle N1, it is possible to integrate the suction tasks of the foreign object Ck and the foreign object Cn.

[0237] In Figure 20(b), the suction task (Cn-N1) when the first suction nozzle N1 is set to operate on foreign object Cn is shown by a thick dashed line. Figure 20(c) shows the suction task for foreign object Cn after this change in aperture size, and the suction tasks for four foreign objects Ck to Cn after the suction task for foreign object Ck has been changed. As a result, the number of overlapping suction tasks in the same time period is now "3," which still exceeds the maximum number of suction nozzles Nx and nx that can be operated simultaneously, which is "2," but the number of overlapping suction tasks in the same time period has been reduced.

[0238] After integrating the suction tasks with the aperture changes of the suction nozzles Nx and nx in step S505, the suction control unit 62 again determines whether or not a conflict state exists (step S506).

[0239] If it is determined in step S506 that there is no conflict, i.e., that the conflict has been resolved (step S506: No), the suction control unit 62 proceeds to step S509, where it determines the suction task after being integrated in step S505, and hands over the suction task to the PLC 8 (step S509). In accordance with the suction task handed over from the suction control unit 62, the PLC 8 performs a suction operation by opening the solenoid valve 7 corresponding to the designated suction nozzle Nx, nx for the designated suction time period (step S510). This completes the suction control process for the detected foreign object Cx.

[0240] If it is determined in step S506 that the conflict state still exists, that is, that the conflict state has not been resolved (step S506: Yes), the suction control unit 62 proceeds to steps S507 to S508 to delete the suction task.

[0241] On the other hand, in step S504, if it is determined that multiple foreign bodies Cx cannot be continuously sucked even if the diameter of the suction nozzles Nx, nx for each foreign body Cx is changed (step S504: No), the suction control unit 62 proceeds to steps S507 to S508 and deletes the suction task.

[0242] When deleting a suction task, the suction control unit 62 first determines the priority of the multiple suction tasks that are in a conflicting state (step S507). In this embodiment, the suction priority is determined based on the weight information D23. Therefore, the suction control unit 62 compares the total values ​​of the weight information D23 of the foreign matter Cx for each suction task, and determines the priority of the suction task in descending order of the total value.

[0243] Thereafter, the suction control unit 62 selects, in descending order of priority according to the priorities determined in step S507, as suction tasks to be executed a number equal to the maximum number of suction nozzles Nx, nx that can be simultaneously operated. Meanwhile, the suction control unit 62 deletes the suction tasks that were not selected (step S508). As a result, the number of suction tasks that overlap in time period becomes the same as the maximum number that can be simultaneously operated, and the conflict state is resolved.

[0244] Following step S508, the suction control unit 62 determines the remaining suction task and hands it over to the PLC 8 (step S509). In accordance with the suction task handed over from the suction control unit 62, the PLC 8 performs a suction operation by opening the solenoid valve 7 corresponding to the designated suction nozzle Nx, nx for the designated suction time period (step S510). This completes the suction control process for the detected foreign object Cx.

[0245] Now, let us refer to Figure 20 again. As shown in Figure 20(c), even after the suction task for foreign object Ck and the suction task for foreign object cn are combined in step S505, the number of suction tasks that overlap in time period is "3," which is greater than the maximum number of suction nozzles Nx and nx that can be simultaneously operated, which is "2." Therefore, in step S507, the total value of the weight information D23 for each suction task is compared.

[0246] In the example of FIG. 20, the magnitude of the weight information D23 calculated from the diagonal length 21 and the standard density value of each of the foreign objects Ck-Cn is in the order Cm > Ck > Cl > Cn. From this, in step S505, the suction task for the foreign object Cn, which has the smallest weight by itself, is combined with the suction task for the foreign object Ck, which is heavier than the foreign object Cl. As a result, in step S507, the magnitude of the total values ​​of the weight information D23 for each suction task is in the order Cm > (Ck + Cn) > Cl. Therefore, in step S507, the suction task for the foreign object Cl is deleted. As a result, there are only two suction tasks with overlapping time periods: the suction task for the foreign objects Ck and Cn (Ck / Cn-N1) and the suction task for the foreign object Cm (Cm-N3). This does not exceed the maximum number of suction nozzles Nx and nx that can be simultaneously operated, which is "2," and therefore the conflict is eliminated.

[0247] Next, another example of exception control will be described with reference to Figures 21 and 22. Figure 21 is a diagram showing an example of six foreign objects Cr to Cw. Figure 21(a) is a diagram showing an example of six foreign objects Cr to Cw and suction nozzles N1 to N3 and n1 to n6 arranged at positions corresponding to these foreign objects Cr to Cw. Figure 22(b) is a diagram showing the initial suction task for the six foreign objects Cr to Cw. Figure 22(c) is a diagram showing the suction task for the six foreign objects Cr to Cw after the task integration in step S502. Figure 22(d) is a diagram showing the suction task for the six foreign objects Cr to Cw after the task integration in step S505.

[0248] When performing exceptional control of the conflict control process for these six particles Cr-Cw, the suction tasks initially calculated in step S305 result in a conflict state in which the suction time periods of the six suction tasks all overlap, as shown in Figure 22(b). Therefore, by integrating the suction tasks without changing the nozzle in steps S501 and S502, the suction tasks for particles Cs and Ct, which are operated by the same second suction nozzle n1, are integrated, and the suction tasks for particles Cu and Cv, which are operated by the same second suction nozzle n4, are integrated. As a result, as shown by the thick solid lines in Figure 22(b), the number of suction tasks with overlapping time periods is still "4," which is greater than the maximum number of suction nozzles Nx and nx that can be simultaneously operated, which is "2."

[0249] Next, by integrating the suction tasks with the calibers of the suction nozzles Nx and nx changed in steps S504 and S505, the foreign matter Cs and foreign matter Ct can be sucked up by the first suction nozzle N1, which is the operating target for the foreign matter Cr. Therefore, the suction tasks for the foreign matter Cs and foreign matter Ct and the suction task for the foreign matter Cr are integrated. The suction task when the operating target for the initial integrated task (Cs / Ct-n1) for the foreign matter Cs and foreign matter Ct is changed to the first suction nozzle N1 is shown by the thick dashed line in Figure 22(c). The integrated task (Cs / Ct-N1) for the foreign matter Cs and foreign matter Ct after the caliber of the operating nozzle has been changed and the suction task for Cr (Cr-N1) are integrated to become the integrated task (Cr / Cs / Ct-N1) shown in Figure 22(d).

[0250] 22(d), the suction tasks for the six foreign objects Cr to Cw are reduced to three: an integrated task for foreign objects Cr, Cs, and Ct (Cr / Cs / Ct-N1), an integrated task for foreign objects Cu and Cv (Cu / Cv-n4), and a suction task for foreign object Cw (Cw-N3). However, the number of suction tasks with overlapping time periods is still "3," which is greater than the maximum number of suction nozzles Nx and nx that can be operated simultaneously, which is "2."

[0251] Therefore, in step S507, a suction priority is determined. In the example of FIGS. 21 and 22, weight information D23 is calculated for each of the impurities Cr through Cw from the diagonal length 21 and the standard density value, and the suction tasks are then integrated. As a result, the order of the total values ​​of the weight information D23 for each suction task is Cw > (Cr + Cs + Ct) > (Cu + Cv). Then, in step S507, the integrated task for the impurities Cu and Cv is deleted. As a result, there are only two suction tasks with overlapping time periods: the suction task for the impurities Cr, Cs, and Ct (Cr / Cs / Ct-N1) and the suction task for the impurity Cw (Cw-N3). This does not exceed the maximum number of suction nozzles Nx and nx that can be simultaneously operated, which is "2," and therefore the conflict is eliminated.

[0252] <7. Arrangement order by nozzle diameter> In this embodiment, as described above, the first nozzle row 51, which is composed of the first suction nozzles Nx with larger diameters, is arranged upstream of the second nozzle row 52, ​​which is composed of the second suction nozzles nx with smaller diameters. That is, the multiple nozzle rows 51, 52, each having multiple suction nozzles Nx, nx for each diameter, are arranged from the upstream side to the downstream side of the transport path in descending order of diameter. In this way, it is preferable that the suction nozzles Nx, nx with larger diameters are located closer to the upstream side of the transport path. This prevents foreign matter exceeding the diameter of a suction nozzle with a relatively small diameter from being sucked in, causing problems such as nozzle blockage.

[0253] 23 and 24 show examples of two foreign bodies Cy and Cz common to both figures, and suction nozzles N1-N3 and n1-n6 positioned corresponding to these foreign bodies Cy and Cz. In the example of Fig. 23, as in the above embodiment, the first suction nozzles N1-N3 with larger apertures are positioned upstream of the second suction nozzles n1-n6 with smaller apertures. On the other hand, in the example of Fig. 24, the second suction nozzles n1-n6 with smaller apertures are positioned upstream of the first suction nozzles N1-N3 with larger apertures.

[0254] 23 and 24, the suction nozzle that is to suck one foreign body Cy is the second suction nozzle n3 with a smaller aperture, and the suction nozzle that is to suck the other foreign body Cz is the first suction nozzle N2 with a larger aperture. The foreign bodies Cy and Cz overlap in their ranges in the transport direction and are close to each other in the width direction.

[0255] 23, when the foreign substance Cz reaches the suction target area of ​​the first suction nozzle N2 and the first suction nozzle N2 performs a suction operation, not only the foreign substance Cz but also the foreign substance Cy is simultaneously sucked and removed by the first suction nozzle N2. Therefore, even if the second suction nozzle n3, which was intended to suck the foreign substance Cy, operates, it cannot suck the foreign substance Cy, but it can suck and remove both the foreign substances Cy and Cz.

[0256] In contrast, in the example of Figure 24, when the foreign matter Cy reaches the suction target area of ​​the second suction nozzle n3 and the suction operation is performed by the second suction nozzle n3, not only the foreign matter Cy but also the foreign matter Cz is sucked into the second suction nozzle n3, and there is a risk that the foreign matter Cz will stick to or clog the second suction nozzle n3.

[0257] If the foreign object Cz ​​sticks to the second suction nozzle n3, the suction performance of the second suction nozzle n3 may be reduced, and the foreign object Cy may remain in the hose 27. Furthermore, after the suction operation of the second suction nozzle n3 is completed, the foreign object Cz ​​may fall onto the conveyor 2, causing the position of the foreign object Cz ​​to shift, which may prevent the first suction nozzle N2 from suctioning the foreign object Cz.

[0258] If the foreign matter Cz clogs the second suction nozzle n3, the second suction nozzle n3 may be unable to perform suction thereafter, and the suction control expected at the time of detection may no longer be established.

[0259] In view of these circumstances, it is preferable that suction nozzles with larger diameters be positioned upstream on the transport path. This prevents the suction action of a suction nozzle nx with a relatively small diameter from attracting foreign matter Cx that is targeted by a suction nozzle Nx with a relatively large diameter. This prevents the foreign matter Cx from sticking to or clogging the suction nozzle nx with a relatively small diameter.

[0260] In the example of FIG. 23, the foreign substance Cy belongs to the suction lane of the first suction nozzle N2, so it is sucked by the first suction nozzle N2, and the suction operation of the second suction nozzle n3 does not need to be performed.

[0261] If we think in general terms, i) There are relatively small foreign objects Cy and relatively large foreign objects Cz, and the minimum nozzle diameters required for suction are different. ii) The suction task of a relatively small foreign object Cy can be integrated with that of a relatively large foreign object Cz ​​by changing the nozzle diameter. iii) A relatively large foreign particle Cz reaches the suction area Vx of a relatively large diameter suction nozzle Nx before a relatively small foreign particle Cy reaches the suction area vx of a relatively small diameter suction nozzle nx. If all three conditions i) to iii) are satisfied, the suction control unit 62 may select the (first) suction nozzle Nx with a relatively large diameter as an exception for a relatively small foreign substance Cy, and may control the (second) suction nozzle nx with a relatively small diameter to refrain from operating (integrate suction tasks).The same applies when nozzle rows are provided for three or more different diameters, such as large, medium, and small.

[0262] This reduces the yield loss when valuable glass material Gl is present around the (second) suction nozzle nx, which has a relatively small diameter. In Figure 23, if suction task integration is not performed, the suction task for foreign matter Cy is assigned to the second suction nozzle n3. However, since the first suction nozzle N2, which is located upstream and has a relatively large diameter, is first activated to perform the suction task for other foreign matter Cz, the foreign matter Cy is likely to be sucked up by the first suction nozzle N2 along with the other foreign matter Cz. If this happens, even if the second suction nozzle n3 is subsequently activated, the foreign matter Cy that should have been sucked up will no longer be present because it has already been sucked up. If glass material Gl is present in the vicinity, it will be unnecessarily sucked up, potentially resulting in a reduction in yield.

[0263] In addition, by doing this, suction tasks that do not contribute to the suction removal of the foreign matter Cy are deleted (integrated), so even if a new foreign matter is detected later, it will not trigger an unnecessary conflict with the foreign matter Cy. This makes it possible to effectively utilize limited suction resources (suction nozzles Nx, nx, which are limited by the maximum number that can be operated simultaneously) to set a suction task for the new foreign matter and suck it up, which is expected to improve the purity of valuable materials.

[0264] <8. Variations> Although the examples of the present invention have been described above, the above-described embodiments are merely examples, and the present invention is not limited to these embodiments.

[0265] In the above embodiment, there is one standard suction cycle time. However, the standard suction cycle time may differ depending on the suction nozzle. For example, the standard suction cycle time may vary depending on the diameter of the suction nozzle. In other words, the standard suction cycle time of the second suction nozzle nx with a smaller diameter may be shorter than the standard suction cycle time of the first suction nozzle Nx with a larger diameter. This minimizes the operating time of the suction nozzle, which is more effective in reducing yield declines.

[0266] In the above embodiment, two types of suction nozzles with different diameters were used: the first suction nozzle Nx and the second suction nozzle nx. However, suction nozzles with more diameters may be used. For example, three types of suction nozzles with large, medium, and small diameters may be used.

[0267] In addition, in the above embodiment, the maximum number of suction nozzles that can be simultaneously operated is set to 2, but this is a hypothetical value adopted for the convenience of explanation and is not limited to this. The maximum number of suction nozzles that can be simultaneously operated can be set to any desired value that is equal to or greater than 2 and does not exceed the total number of suction nozzles Nx, nx. The maximum number of suction nozzles that can be simultaneously operated may also be set to 1. Even if any arbitrary value is adopted for the maximum number of suction nozzles that can be simultaneously operated, the generality of the explanation in the above embodiment will not be lost.

[0268] In addition, in the above embodiment, various setting and registration values ​​related to the control of the sorting system 1 are shown, but it is not necessary to set all of these setting and registration values. Furthermore, setting values ​​such as thresholds, conditions, and target values ​​not described in the above embodiment may be set and registered and used for controlling the sorting system 1.

[0269] In particular, the determination of the suction priority is not limited to the above embodiment, and for example, different indices may be used to determine the suction priority in the conflict control process (without exception control) and the conflict control process (with exception control).

[0270] Furthermore, the reliability lower limit value for the reliability information D14 and the size lower limit value for the area information D11 may be variable depending on the production target values ​​(D) such as the target yield rate and target purity, and may be changed to maximize the yield rate, provided that the target purity can be achieved. On the other hand, if the target purity cannot be achieved, these values ​​may be changed to prioritize the valuable material purity regardless of the yield rate.

[0271] Furthermore, a size lower limit value and a reliability lower limit value may be set for each type information D13. For example, if the tolerance for remaining in valuables for a particular type of foreign matter is low, the size lower limit value for the type of foreign matter with the low tolerance may be set small (or set to 0). Furthermore, when determining the suction priority, multiple pieces of information may be combined, and multiple suction priority determination conditions (C) may be weighted and used so that types of foreign matter Cx with a low tolerance for remaining are preferentially sucked. For example, when determining the suction priority, the type information D13 and weight information D23 may be combined to determine the priority, with the type information D13 being given priority over the weight information D23.

[0272] In the above embodiment, the shape of the area graphic represented by the area information D11 is a rectangle, but the present invention is not limited to this. The shape of the area graphic represented by the area information D11 can be any geometric shape. The shape of the area graphic may be, for example, a rectangle, a polygon, a circle, or any other desired planar shape, or may be a square prism, a polygonal prism, a cylinder, a sphere, or any other desired three-dimensional shape. In this case, the bounding box may be transformed into the area graphic of the desired shape by a predetermined calculation.

[0273] The area information D11 also includes values ​​of geometric quantities of the geometric figure, which is the shape of the area graphic. The area information D11 can include, for example, the lengths of the sides, diagonals, circumference, diameter, etc. of the geometric figure, as well as the area or volume of the geometric figure calculated based on these.

[0274] In the above embodiment, the size of the area information D11 is the diagonal length D21 of the bounding box indicated by the area information D11. However, the present invention is not limited to this. Instead of the diagonal length D21, the size of the area information D11 may be a metric value related to the shape of the desired area graphic represented in the area information D11. Furthermore, the size may be the position information D12 or the center coordinates (center of gravity coordinates) of the desired geometric graphic represented in the area information D11.

[0275] Furthermore, the area information D11 may be the outline of the area where the foreign object Cx is present, estimated based on the results of machine learning of feature quantities such as the color and texture of the foreign object Cx using a trained model M. In this case, the area where the foreign object Cx is present can be identified in more detail than when the foreign object Cx is trimmed using an area graphic. This allows for more accurate estimation of size-related identification information such as the area information 21 of the foreign object.

[0276] However, when estimating the outer shape of the area in which the foreign matter Cx exists, it is considered that the amount of calculation required to calculate the secondary identification information D2, such as the size of the area information D11 (corresponding to the diagonal length D21), the area information D22, and the weight information D23, will be greater than when the area graphic is a rectangular bounding box. This increases the calculation time, which may exceed the predetermined response time required by the sorting system 1 (impairing responsiveness). For this reason, in the sorting system 1 that needs to simultaneously detect a large number of foreign matters Cx and quickly determine suction tasks, it is preferable to provide a trained model M that has been machine-learned to be able to estimate the area information D11 using a bounding box.

[0277] Furthermore, when using feature quantities such as the color and texture of the foreign matter Cx, it is preferable to perform large-scale and high-speed foreign matter detection using rectangular area figures, and, if necessary, to reduce the amount of calculation, for example, by calculating the area occupied by the color of the foreign matter Cx (color area).

[0278] Alternatively, the area occupied by the foreign object Cx may be directly estimated using the feature quantities such as color and texture of the foreign object Cx, and then a figure of a desired shape such as a bounding box may be superimposed on the estimated area occupied by the foreign object Cx.

[0279] In the resource recycling process, depending on the characteristics of the foreign matter, some foreign matter has a relatively small impact on the final quality of the valuable material (including cases other than glass material GI). For this reason, 100% removal of such foreign matter is not necessarily required. Increasing the purity of the valuable material reduces the yield rate, and there is a trade-off between the two. Therefore, the size and reliability lower limits for operation suppression should be determined according to quantitative target values ​​for product quality and yield, such as the target yield rate and target purity of the valuable material, i.e., the production target value (D).

[0280] In the above embodiment, the positions of the suction nozzles Nx, nx relative to the conveyor 2 are fixed. However, the suction nozzles Nx, nx may be movable. For example, by making the suction nozzles Nx, nx movable in the vertical direction, the suction nozzles can be operated close to each foreign object Cx according to the standard height of the objects to be sorted Ob, thereby saving air for operating the compressor 25.

[0281] However, it is preferable that the suction nozzles Nx, nx are fixed at least in the horizontal direction, which is effective in reducing the cost of the device and the number of steps required for setting up the suction nozzles Nx, nx compared to using a movable robot arm or the like for the suction nozzles Nx, nx.

[0282] In the above embodiment, the suction nozzles Nx, nx that suck and remove the foreign matter Cx are used as the foreign matter Cx removal means. However, the present invention is not limited to this. Alternatively, a pneumatic ejector or the like may be used as the foreign matter Cx removal means, which blows gas onto the foreign matter Cx to remove it by blowing it away.

[0283] In the above embodiment, the removal capabilities of all suction nozzles Nx, nx are generally constant. However, the removal capabilities of the foreign matter Cx removal means may be variable. For example, multiple types of removal means with different outputs may be provided, or the output of the same removal means may be variable.

[0284] In the device configuration and each step according to the present invention, the elements appearing in the above-described embodiments and modifications may be combined as appropriate within the scope of not causing any contradiction. [Industrial Applicability]

[0285] The present invention can be used for resource recycling to recover transparent valuables such as glass materials from sorting objects such as crushed solar panels, as well as for recovering valuable metals and other valuables from various types of waste. [Explanation of symbols]

[0286] 1: Sorting system 2: Conveyor 4: Camera 5: Suction part 6: Information processing device 7: Solenoid valve 9: Food 10: Image capture range 11: LED lighting 16: First polarizing plate 44: Second polarizing plate 51: First nozzle row 52: Second nozzle row 61: Detection unit 62: Suction control unit 63: Storage section Cx,Ca~Cz: Foreign matter D1: Original identification information D11: Area Information D12: Location information D13: Type information D14: Reliability information D2: Secondary identification information D22: Area information D23: Weight information Gl: Valuables, glass materials M: Trained model Nx: First suction nozzle nx: Second suction nozzle Ob: Object to be sorted

Claims

1. an imaging means for imaging a sorting target object on a conveyance path, which includes a mixture of transparent valuables and opaque foreign objects, to obtain imaging data of the imaging range; a plurality of suction nozzles arranged downstream of the imaging means; an information processing device communicably connected to both the suction nozzle and the imaging means; A valuable resource sorting system that detects and removes the foreign matter from the sorting object, The information processing device includes: When the imaging data is input, a trained model is machine-learned to be able to detect the presence of the foreign matter contained in the object to be sorted. Equipped with The information processing device is capable of controlling the operation of the suction nozzle based on the detection results of the trained model.

2. When the imaging data is input, the trained model can detect the presence of the foreign matter contained in the sorting object and estimate identification information that identifies the foreign matter, The identification information is Area information indicating the range of the foreign matter on the transport path. The valuable resource sorting system according to claim 1, comprising:

3. When the imaging data is input, the trained model can detect the presence of the foreign matter contained in the sorting object and estimate identification information that identifies the foreign matter, The identification information is information on the type of the foreign matter; Area information indicating the area occupied by the foreign matter on the transport path; and Reliability information indicating the accuracy of the foreign object detection result The valuable resource sorting system according to claim 1, characterized in that it includes at least one of the following:

4. Upon detection of the foreign object, If the type information of the foreign matter corresponds to a predetermined type, When the size of the area information of the foreign object is below a predetermined area lower limit value, and If the reliability information of the foreign object is below a predetermined reliability lower limit value, 4. The valuable resource sorting system according to claim 3, wherein the operation of the removing means for the foreign matter is postponed if any of the following conditions is met:

5. When the imaging data is input, the trained model can detect the presence of the foreign matter contained in the sorting object and estimate identification information that identifies the foreign matter, The identification information includes at least Position information indicating the position of the foreign object on the transport path Including, 5. The valuable resource sorting system according to claim 1, wherein the information processing device selects the suction nozzle corresponding to the position information as an operation target.

6. A valuable resource sorting method for removing foreign matter from a sorting target containing a mixture of valuable resources and foreign matter using a plurality of suction nozzles, a) capturing an image of the object to be sorted on a conveying path to acquire image data; b) detecting the foreign object included in the imaging data using a trained model; c) selecting one of the suction nozzles as an operation target based on the detection result of the trained model; A method for sorting valuable materials, including:

Images