Screening apparatus and screening method

Abstract

To screen a crushed piece of a predetermined material by reducing a treatment load in screening treatment.SOLUTION: A screening apparatus for screening a crushed piece of a predetermined material from multiple crushed pieces obtained by crushing a vehicle structure comprises: a first optical image photographing section for photographing an optical image of a crushed piece; an infrared image photographing section for photographing an infrared image of a crushed piece; and a treatment section for screening a crushed piece of a predetermined material on the basis of an optical image and an infrared image. The treatment section screens a crushed piece of a predetermined material by performing an image analysis of an optical image limited to an analysis area corresponding to a specific area in an optical image when detecting a specific area causing a temperature contrast having a characteristic shape in an infrared image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a sorting device and a sorting method. [Background technology]

[0002] In recent years, efforts to significantly reduce waste generation through waste prevention, reduction, recycling, and reuse have been gaining momentum. To achieve this, research and development is being conducted into the recycling of metal materials that make up car bodies, such as steel and aluminum.

[0003] Patent Document 1 discloses a sorting device that sorts mild steel based on the shape of crushed pieces. The sorting device disclosed in Patent Document 1 identifies as wrinkles portions of the surface of the crushed pieces where the difference in index value compared to the surrounding area is equal to or greater than a threshold, based on surface information that indicates a height index value corresponding to the position of the crushed pieces, and sorts the crushed pieces based on the wrinkle ratio, which is the proportion of the surface of the crushed pieces that are wrinkled. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6726753 specification Summary of the Invention [Problem to be solved by the invention]

[0005] The sorting device disclosed in Patent Document 1 uses crushed mild steel and uncrushed high tensile steel as sorting components based on the percentage of wrinkles on the surface of the crushed pieces. Since image analysis is required to determine the percentage of wrinkles from the overall image of the crushed pieces, the processing load for performing the sorting process is high.

[0006] In view of the above-mentioned problems, the present invention aims to provide a technology that is advantageous for reducing the processing load in the sorting process and sorting out crushed pieces of a specified material. [Means for solving the problem]

[0007] A sorting device according to one aspect of the present invention is a sorting device that sorts out crushed pieces of a predetermined material from a plurality of crushed pieces obtained by crushing a vehicle body structure, the sorting device comprising: a first optical image capturing unit that captures an optical image of the fragments; an infrared image capturing unit that captures an infrared image of the fragments; a processing unit that sorts out fragments of the predetermined material based on the optical image and the infrared image, When the processing unit detects a specific area in the infrared image where a temperature contrast with a characteristic shape occurs, it performs image analysis of the optical image limited to an analysis area corresponding to the specific area, thereby sorting out fragments of the specified material.

[0008] Another aspect of the present invention is a sorting method for a sorting device that sorts crushed fragments of a predetermined material from a plurality of crushed fragments obtained by crushing a vehicle body structure, the method comprising: A step of taking an optical image of the fragments by a first optical image taking unit; a step of capturing an infrared image of the fragments by an infrared image capturing unit; and a processing step of sorting out fragments of the predetermined material based on the optical image and the infrared image, In the processing process, when a specific area in the infrared image where a temperature contrast with a characteristic shape occurs is detected, image analysis of the optical image is performed limited to an analysis area corresponding to the specific area, thereby sorting out fragments of the specified material. [Effects of the Invention]

[0009] According to the present invention, it is possible to reduce the load in the sorting process and sort out crushed pieces of a predetermined material. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a sorting device according to an embodiment. [Figure 2] FIG. 2 is a diagram showing an outline of processing by the sorting device according to the embodiment. [Figure 3] 10A and 10B are diagrams showing a first example of detection of a specific area in an infrared image and an analysis area in an optical image. [Figure 4] 10A and 10B are diagrams showing a second example of detection of a specific area in an infrared image and an analysis area in an optical image. [Figure 5] 10A and 10B are diagrams showing a third example of detection of a specific area in an infrared image and an analysis area in an optical image. [Figure 6] 10A and 10B are diagrams showing a fourth example of detection of a specific area in an infrared image and an analysis area in an optical image. [Figure 7] 1A and 1B are diagrams illustrating an infrared image and optical images taken from two directions. [Figure 8] 10A and 10B are diagrams illustrating a comparison example between an optical image and a radiographic image. [Figure 9] 10 is a diagram illustrating the relationship between the distance in the radiation irradiation direction and the thickness of fragments in the irradiation direction. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention as claimed, and not all combinations of features described in the embodiments are necessarily essential to the invention. Two or more of the features described in the embodiments may be arbitrarily combined. Furthermore, the same reference numerals are used for the same or similar components, and redundant explanations will be omitted.

[0012] (Configuration example of sorting device 100) 1 is a diagram showing an example of the configuration of a sorting device 100 according to an embodiment. The sorting device 100 sorts out fragments of a predetermined material from a plurality of fragments (OB1...OB5, etc.) obtained by shredding a vehicle body structure using a crusher (not shown) (hereinafter also referred to as a shredder). In the following description, the fragments will be collectively referred to as fragments OB.

[0013] In this embodiment, the target is the crushed pieces OB obtained after crushing the vehicle body structure in a crusher and removing non-ferrous metals by magnetic sorting. The shapes of the crushed pieces OB can be classified into the following three types, for example, from the viewpoint of thickness and strength.

[0014] (a) Crumpled at small intervals (for example, mild steel plate with a thickness of 1.0 mm or less, essentially ultra-low carbon steel for outer panels) (b) Crushed at large intervals (for example, mild steel plates with a thickness of 2.0 mm or more, essentially hot-rolled steel plates used as reinforcing members) (c) Flat and uncrushed (high-strength steel (hereinafter referred to as high-tensile steel) with a plate thickness of 1.0 mm or more, body frame parts) Furthermore, copper wire (Cu wire), brazing parts containing Cu, structural steel such as bolts, etc. may also be mixed in as foreign matter with the crushed pieces OB.

[0015] The sorting device 100 of this embodiment has a configuration for acquiring image information of the crushed fragments OB, including an infrared image capturing unit 12A (thermo camera) that captures infrared images, a first optical image capturing unit 12B (first visible light camera) that captures optical images, a second optical image capturing unit 12C (second visible light camera), and a radiation capturing unit (radiation generating unit 13A, radiation detecting unit 13B). These are collectively referred to as "image information capturing units (12A, 12B, 12C, 13A, 13B)."

[0016] The sorting device 100 has an image information acquisition unit (12A, 12B, 13A, 13B) that acquires image information of the broken fragments OB, a processing unit 110 that processes the information acquired by the image information acquisition unit, and a control unit 120 that controls the sorting mechanism 14A based on the judgment result of the processing unit 110.

[0017] The processing unit 110 of this embodiment is capable of performing inference processing using a trained machine learning model 150. In machine learning, a prepared infrared image is used as an input to the learning model, and machine learning is performed to output a selection determination result as an output, thereby configuring the trained machine learning model 150.

[0018] In machine learning using infrared images, images containing specific regions with characteristic shapes where temperature contrasts occur (regions with characteristic shapes and different temperature distributions) are added as input data to the learning model as infrared images. Here, specific regions can include, for example, regions where linear temperature distributions suddenly change, such as blazed parts, edge faces of broken pieces, regions in the infrared image that contain background parts different from the broken pieces, regions where copper wires (Cu wires) are tangled, regions that contain fasteners such as bolts and nuts, etc.

[0019] In machine learning, information on the temperature distribution of the broken fragments OB in an infrared image may be used as an input image (input data). Differences in temperature distribution over time after heating the broken fragments OB may be used as information on the temperature distribution of the broken fragments OB. Alternatively, differences in brightness due to temperature changes detected in an infrared image (differences in emissivity calculated from brightness) may be used as information on the temperature distribution of the broken fragments OB. Alternatively, differences in thermal conduction patterns or differences in thermal conductivity that can be calculated from changes in brightness due to temperature changes detected in an infrared image may be used as information on the temperature distribution of the broken fragments OB. Alternatively, differences in reflectance or brightness information when irradiated with light of a specific wavelength may be used as information on the temperature distribution of the broken fragments OB. Machine learning may be performed in advance by combining these data as input images (input data), and a trained machine learning model 150 may be constructed.

[0020] By performing such machine learning, it is possible to locally limit the region to be analyzed, thereby reducing the computational load. Note that when performing machine learning, the machine learning model 150 may be configured to prepare an optical image for learning, perform machine learning, and output a determination result of whether or not a specific region exists.

[0021] In the inference phase, an infrared image of the broken pieces OB transported by the transport mechanism 11, which will be described later, becomes input data for the machine learning model 150. If the infrared image contains a specific region, the processing unit 110 outputs a determination result. On the other hand, if the infrared image does not contain a specific region, an inference process based on the output data of the machine learning model 150 can be used to sort out the broken pieces of ultra-low carbon steel for outer panels (hereinafter also referred to as the broken pieces to be sorted) as classified in (a) above.

[0022] The conveying mechanism 11 may be configured to be capable of conveying a plurality of crushed pieces OB, and typically uses a belt conveyor that drives a belt 11A using a drive pulley 11B and a driven pulley 11C. A drive source such as a motor (not shown) is connected to the drive pulley 11B, and the rotational power of the drive source is transmitted to the drive pulley 11B, allowing the conveying mechanism 11 to convey the crushed pieces OB in a predetermined direction in the direction of the arrow. The belt 11A forms a conveying path for conveying each crushed piece OB, and is capable of conveying various crushed pieces OB1, OB2, and OB3 obtained by crushing a vehicle body structure using a crusher (shredder) in the direction of the arrow.

[0023] In FIG. 1, the left side of the paper is the upstream side in the conveying direction, and the right side of the paper is the downstream side in the conveying direction. The conveying direction is the X direction, the vertical direction intersecting the X direction is the Z direction, and the direction perpendicular to the paper surface intersecting the XZ direction is the Y direction. The crushed pieces OB discharged from the shredder are supplied to the upstream side of the conveying mechanism 11, which conveys each crushed piece OB from upstream to downstream. The crushed pieces OB are photographed by image information acquisition units (12A, 12B, 12C, 13A, 13B) on the conveying path conveyed by belt 11A. Image information photographed by the image information acquisition units (12A, 12B, 12C, 13A, 13B) is input to the processing unit 110. Note that the photographing by the second optical image photographing unit 12C and the radiography units (13A, 13B) is auxiliary, and photographing may be performed according to photographing instructions from the processing unit 110.

[0024] The infrared image capturing unit 12A (thermo camera) is configured to capture images of the broken pieces OB on the belt 11A in the infrared region, and is used to detect the amount of infrared radiation emitted from the broken pieces OB. Image information (infrared image) captured by the infrared image capturing unit 12A is input to the processing unit 110. The processing unit 110 can acquire information on the temperature distribution of the broken pieces OB based on the image information (infrared image). For example, when a specific region is included in the infrared image, the processing unit 110 may acquire information on the temperature distribution of the specific region extracted by the output of the machine learning model 150.

[0025] The first optical image capturing unit 12B (visible light camera) is configured to capture images of the broken pieces OB on the belt 11A in the visible light range. The first optical image capturing unit 12B may be, for example, a camera configured with a CCD / CMOS image sensor. Image information (optical image) captured by the first optical image capturing unit 12B is input to the processing unit 110. The processing unit 110 can acquire appearance information of the broken pieces OB based on the image information (optical image). When temperature information of a specific region is acquired in the infrared image, the processing unit 110 can perform image analysis based on appearance information limited to the specific region, rather than the entire optical image.

[0026] The processing unit 110 also acquires position information of the fragments OB on the belt 11A based on the image information (optical image) acquired from the first optical image capturing unit 12B. The position information of the fragments OB is linked to the image information (infrared image) of the infrared image capturing unit 12A, the image information (optical image) of the second optical image capturing unit 12C, and the radiation image information of the radiation capturing units (13A, 13B). Furthermore, when switching the sorting destination (15A or 15B) of the sorting mechanism 14A, the fragments OB to be sorted are identified based on the position information of the fragments OB on the belt 11A.

[0027] The second optical image capturing unit 12C is configured to capture an auxiliary optical image. The second optical image capturing unit 12C is located downstream of the first optical image capturing unit 12B and is positioned so that it can capture images of the fragments OB at an angle of view different from that of the first optical image capturing unit 12B.

[0028] When it is determined that the accuracy of the image analysis of the optical image needs to be improved, the processing unit 110 outputs an image capture instruction signal to the second optical image capture unit 12C. The second optical image capture unit 12C is configured to capture an optical image of the broken fragments OB based on the reception of the image capture instruction signal. The captured image information (optical image) is input to the processing unit 110. The processing unit 110 can perform more accurate image analysis by using multiple optical images with different angles of view.

[0029] 1 shows two examples of the first optical image capturing unit 12B and the second optical image capturing unit 12C, but the present invention is not limited to this example and two or more optical image capturing units may be provided. Furthermore, the position of the second optical image capturing unit 12C is not limited to the upper side of the belt 11A and may be disposed, for example, in the lateral direction of the belt 11A (perpendicular to the paper surface).

[0030] The radiography unit has a radiation generation unit 13A that irradiates the fragments OB with radiation, and a radiation detection unit 13B that detects the radiation that has passed through the fragments OB.

[0031] The radiation generating unit 13A starts irradiating radiation in accordance with an irradiation start command from the processing unit 110. The radiation is typically X-rays, but may also be gamma rays or the like. The radiation irradiated from the radiation generating unit 13A passes through the fragments OB and enters the radiation detecting unit 13B. In addition, the radiation generating unit 13A stops irradiating radiation in accordance with an irradiation stop command from the processing unit 110.

[0032] As an example configuration, the radiation detection unit 13B may include a radiation detection panel having a pixel array in which pixels that detect radiation are arranged two-dimensionally, a conversion circuit that acquires charges corresponding to the irradiated dose, and an image generation unit that generates a radiation image based on the charge information. Note that a line sensor in which pixels are arranged one-dimensionally may be used instead of the radiation detection panel having a two-dimensional pixel array. The image generation unit generates a radiation image based on charge information corresponding to the radiation incident on the radiation detection panel and transmits the generated radiation image to the processing unit 110.

[0033] Differences in the thickness of the material constituting the fragments OB and the state of crushing (for example, crushed into a ball, wrinkled without being crushed, etc.) result in differences in the dose of radiation that passes through the fragments OB, and differences in dose can cause shading in the radiological image showing the interior of the fragments OB. Based on the radiological image, the processing unit 110 can determine the thickness of the fragments OB or the distribution of thickness inside (in the cross-sectional direction) the crushed fragments OB.

[0034] (Configuration of processing unit 110) The processing unit 110 may be configured with a dedicated circuit such as a PLD (Programmable Logic Device) such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). Alternatively, the processing unit 110 may be configured with a combination of a general-purpose processing circuit such as a processor and a storage circuit such as a memory. In this case, the general-purpose processing circuit may execute a program stored in the storage circuit to realize the functions of the processing unit 110.

[0035] The processing unit 110 is capable of performing material sorting processing based on inference processing using deep learning, neural networks, etc. The processing unit 110 is capable of sorting out predetermined materials from the crushed pieces OB by applying a trained machine learning model 150 to an image (infrared image) acquired from the infrared image capturing unit 12A as an input image. Various materials can be selected as the material to be sorted, but it is also possible to select steel plates (ultra-low carbon mild steel plates) having a thickness of a predetermined plate thickness or less (for example, 1 mm or less).

[0036] The processing unit 110 transmits determination information indicating the material sorting results and position information of the crushed pieces OB that were selected to the control unit 120. Based on the position information of the crushed pieces OB and the determination information received from the processing unit 110, the control unit 120 transmits an operating signal to switch the sorting destination (15A or 15B) of the sorting mechanism 14A.

[0037] Based on a movement signal received from the control unit 120, the sorting mechanism 14A adjusts the angle of the sorting panel 14B to switch the sorting destination of the crushed pieces OB transported by the belt 11A to the collection unit 15A or 15B. The example in Figure 1 shows a state in which crushed pieces OB4 sorted as a predetermined material (ultra-low carbon mild steel plate) have been sorted and collected in the collection unit 15B. And, a state in which crushed pieces OB5 of a material other than the predetermined material have been sorted and collected in the collection unit 15A.

[0038] In the configuration example of the sorting mechanism 14A shown in FIG. 1, the sorting mechanism 14A changes the angle of the sorting panel 14B based on a movement signal, but other mechanisms that can achieve the same function may be used. For example, an air blow system or a robot arm system may be used. Boxes, trays, etc. are typically used for the collection units 15A and 15B. In the configuration example shown in FIG. 1, the number of collection units 15A and 15B is two, but three or more may be used.

[0039] (Outline of processing by sorting device 100) 2 is a diagram showing an overview of the processing of the sorting device 100 according to the embodiment. In step S21, the first optical image capturing unit 12B captures an image of the broken pieces OB on the belt 11A. The image information captured by the first optical image capturing unit 12B is input to the processing unit 110. The processing unit 110 identifies the position information of the broken pieces OB on the belt 11A based on the image information acquired from the first optical image capturing unit 12B (S71).

[0040] In step S31, the infrared image capturing unit 12A captures an image of the broken pieces OB on the belt 11A. The image information captured by the infrared image capturing unit 12A is input to the processing unit 110.

[0041] In step S32, the processing unit 110 selects fragments of a predetermined material based on the optical image and the infrared image. When the processing unit 110 detects a specific region in the infrared image where a temperature contrast with a characteristic shape occurs, the processing unit 110 selects fragments of the predetermined material by performing image analysis of the optical image limited to an analysis region corresponding to the specific region. The processing unit 110 detects a region in the infrared image where the temperature distribution trend of the fragments differs as the specific region, and performs image analysis limited to the region where the temperature distribution trend differs as the analysis region in the optical image. The processing unit 110 may determine whether the fragments are fragments of a predetermined material based on color information obtained by image analysis. The processing unit 110 may determine whether the fragments are fragments of a predetermined material based on the shape of the fragments obtained by image analysis.

[0042] In the inference process using infrared images, image information (infrared images) acquired from the infrared image capturing unit 12A is input to the trained machine learning model 150, and the processing unit 110 can acquire the sorting results of the broken pieces OB as determination information as an output of the machine learning model 150. The time from processing by the shredder until the infrared image is captured by the infrared image capturing unit 12A may be the elapsed time after heating. When shredding the vehicle body structure with a shredder and subsequently capturing an infrared image, the broken pieces are heated to a high temperature by processing heating, so there is no need to heat the broken pieces. Furthermore, when sorting broken pieces that have been cooled to room temperature, they may be reheated using infrared rays, high frequency waves, or other methods.

[0043] Furthermore, the processing unit 110 may classify the shape of the broken pieces OB based on an inference process based on the infrared image and the external characteristics of the broken pieces OB based on the optical image. Specifically, the processing unit 110 uses the trained machine learning model 150 to determine whether a specific region has been detected in the broken pieces OB from the infrared image acquired from the infrared image capturing unit 12A. If no specific region has been detected in the infrared image, the broken pieces OB may be determined by an inference process using the entire infrared image.

[0044] Furthermore, if a specific region is detected in the infrared image, the processing unit 110 may acquire the temperature distribution, etc. in the specific region and perform inference processing limited to the specific region. In this case, the processing unit 110 may determine the presence of broken fragments OB by combining the inference processing of the specific region in the infrared image with image analysis of the specific region in the optical image. When performing image analysis of the specific region in the optical image, the processing unit 110 may perform image analysis of the specific region in the optical image using the optical image captured in step S21. If temperature information of the specific region in the infrared image is acquired, the processing unit 110 may perform image analysis limited to the specific region, rather than the entire optical image.

[0045] FIG. 3 shows a first example of detection of a specific region in an infrared image and an analysis region in an optical image. In the first example of detection in FIG. 3, the processing unit 110 detects a region in the infrared image where the temperature distribution of the fragments is linear as the specific region, and performs image analysis by limiting the analysis region in the optical image to the linear temperature distribution region. FIG. 3(A) shows an example of detection of a specific region (a linear temperature sudden change region), which includes a braze portion 301 included in the fragments OB and a background portion 302 on the belt 11A on which the fragments OB are placed. The braze portion 301 shown in FIG. 3(A) is a portion brazed and joined using, for example, a Cu-based brazing filler metal, and the braze portion 301 is linearly distributed in the fragments OB. A region of low temperature distribution is also linearly distributed in the background portion 302.

[0046] FIG. 3(B) is a diagram showing analysis regions limited to specific regions (blaze portion 301, background portion 302) in the optical image captured in S21. Analysis region 303 is an analysis region corresponding to the specific region (blaze portion 301) in the infrared image. Analysis region 304 is an analysis region corresponding to the specific region (background portion 302) in the infrared image. Through image analysis of analysis region 303, processing unit 110 detects the color of copper (Cu) in the area with low temperature distribution, and determines that analysis region 303 is a braze portion brazed using a Cu-based brazing filler metal. Furthermore, through image analysis of analysis region 304, processing unit 110 determines that the area with low temperature distribution is an empty area in the optical image, and determines that analysis region 304 is an image of an area unrelated to the broken fragments OB.

[0047] FIG. 4 is a diagram showing a second detection example of a specific region in an infrared image and an analysis region in an optical image. In detection example 2 of FIG. 4, processing unit 110 also detects a region in the infrared image where the temperature distribution of the fragments is linear as the specific region, and performs image analysis by limiting the analysis region in the optical image to the region where the temperature distribution is linear. FIG. 4(A) is a diagram showing a detection example of an end face 401 of fragment OB as a detection example of a specific region (a linear temperature sudden change portion). End face 401 shown in FIG. 4(A) has a linear distribution of low temperature regions, similar to blazed portion 301.

[0048] 4(B) is a diagram showing an analysis area limited to a specific area in the optical image captured in S21. Analysis area 402 is an analysis area corresponding to the specific area (end face 401) in the infrared image. Based on the shape and color information obtained by image analysis of analysis area 402, processing unit 110 determines that the area with low temperature distribution is not copper (Cu) but the end face of broken fragment OB.

[0049] FIG. 5 illustrates a third example of detection of a specific region in an infrared image and an analysis region in an optical image. In the third example of detection in FIG. 5, the processing unit 110 detects, as the specific region, a region in the infrared image where a fibrous object exists, where the temperature distribution of the fragments differs from that of the surrounding area. The processing unit 110 then performs image analysis by limiting the analysis region in the optical image to the region where the fibrous object exists. FIG. 5(A) illustrates, as an example of detection of a specific region, a detection example of a fibrous object 501 whose temperature is lower than the surrounding temperature. The fibrous object 501 illustrated in FIG. 5(A) illustrates a state in which copper wires (Cu wires) are tangled. FIG. 5(B) illustrates an analysis region limited to the specific region (fibrous object 501) in the optical image captured in S21. The analysis region 502 corresponds to the specific region (fibrous object 501) in the infrared image. The processing unit 110 performs image analysis of the analysis region 502 and determines that the specific region (fibrous object 501) is not a resin fiber but a copper wire (Cu wire) based on the color information obtained by the image analysis.

[0050] FIG. 6 shows a fourth example of detection of a specific region in an infrared image and an analysis region in an optical image. In the fourth example of detection in FIG. 6, the processing unit 110 detects a region in the infrared image where a polygonal object exists, where the temperature distribution of the fragments differs from the surrounding area, as the specific region, and performs image analysis by limiting the analysis region in the optical image to the region where the polygonal object exists. FIG. 6(A) shows an example of detection of polygonal (e.g., hexagonal) objects 601 and 602, which have a temperature distribution tendency different from the surrounding temperature, as a detection example of a specific region. FIG. 6(B) shows an analysis region limited to the specific regions (hexagonal objects 601 and 602) in the optical image captured in S21. Analysis regions 603 and 604 are regions corresponding to the specific regions (hexagonal objects 601 and 602) in the infrared image, respectively. The processing unit 110 determines that the specific regions (hexagonal objects 601, 602) are bolts through image analysis of the analysis regions 603, 604. Broken pieces OB containing foreign matter such as bolts and copper wires are excluded from the broken pieces to be sorted out.

[0051] When a specific region is detected in the infrared image, image analysis can be performed only on the specific region in the optical image, thereby limiting the analysis region locally, reducing the computational load and improving the accuracy of the determination. Furthermore, even if there is dirt or paint residue on the surface of the fragments OB, the determination accuracy can be improved without being affected by these, while saving computer resources.

[0052] In the image analysis of step S32, if it is determined that the predetermined judgment accuracy cannot be obtained by analyzing the optical image captured by the first optical image capturing unit 12B alone, the processing unit 110 outputs an image capturing instruction signal to the second optical image capturing unit 12C.

[0053] In step S41, the second optical image capture unit 12C captures an optical image of the broken fragments OB based on the reception of the image capture instruction signal. The position information of the broken fragments OB to be captured on the belt 11A has been identified, and the broken fragments OB are the same as the broken fragments (e.g., OB3 in FIG. 1) captured on the upstream side of the belt 11A by the first optical image capture unit 12B and the infrared image capture unit 12A. The second optical image capture unit 12C captures the broken fragments OB at a different angle of view from that of the first optical image capture unit 12B, and transmits the captured image information (optical image) to the processing unit 110.

[0054] The processing unit 110 may classify the shape of the fragments OB based on inference processing based on the infrared images and the external appearance characteristics of the fragments OB based on a plurality of optical images with different angles of view.

[0055] 7A and 7B are diagrams illustrating an infrared image and optical images captured from two directions. 701 in Fig. 7B is an infrared image 701 captured by the infrared image capturing unit 12A, 702 in Fig. 7A is an optical image 702 captured by the first optical image capturing unit 12B, and 703 in Fig. 7C is an optical image 703 captured by the second optical image capturing unit 12C. The arrow direction shown as conveying direction 704 is the conveying direction of belt 11A in Fig. 1, and the optical image 702, infrared image 701, and optical image 703 are captured in this order along the path of the fragments OB being transported by belt 11A from the upstream side to the downstream side.

[0056] High-temperature areas are present randomly in infrared image 701, and inference processing based on infrared image 701 may not be able to distinguish whether the high-temperature areas represent the shape of the fragments or the remaining paint. The optical image captured by first optical image capture unit 12B is an optical image of one side of fragment OB. If processing unit 110 determines that a predetermined level of accuracy cannot be achieved by combining the analysis results of infrared image 701 and optical image 702, it outputs an image capture instruction signal to second optical image capture unit 12C to capture optical image 703 of a different side. Image analysis of optical image 702 and optical image 703 allows image information to be obtained from optical images captured in different directions for the area identified as a high-temperature area in infrared image 701.

[0057] The processing unit 110 classifies the shape of the broken pieces OB based on inference processing based on the infrared image 710 and the external characteristics of the broken pieces OB based on the optical images 720 and 730, and can determine that the broken pieces OB in Figure 7 represent the shape of the broken pieces OB and that the high-temperature part is inside the crushed broken pieces OB. In this case, the processing unit 110 determines that the broken pieces are thin soft iron (broken pieces to be sorted).

[0058] In the image analysis process of step S32, for example, if the end face 401 of the fragment OB is visible as the analysis area 402 corresponding to the specific area (end face 401) as shown in FIG. 4, detailed determination based on the plate thickness is possible, and therefore the processing unit 110 proceeds to step S51. That is, if it is determined that information on the thickness of the fragments is necessary through image analysis of the optical image in order to sort out fragments of a predetermined material, the processing unit 110 proceeds to step S51. Note that the image analysis of the optical image is not limited to image analysis of the optical image captured by the first optical image capturing unit 12B. If it is determined that information on the thickness of the fragments is necessary through image analysis of the optical image captured by the second optical image capturing unit 12C, the processing may proceed to step S51.

[0059] In step S51, the processing unit 110 outputs an irradiation start command to the radiation generation unit 13A to start radiography. The radiation detection unit 13B generates a radiographic image of the fragments OB and transmits it to the processing unit 110.

[0060] In step S52, the processing unit 110 measures the thickness of the fragments OB based on the radiographic images. Here, as a specific example of the radiographic images, FIG. 8 shows a comparison example between optical images (OM81-84) and radiographic images (XM81-84). Optical image OM81 is an optical image of a thin plate crushed into a spherical shape, and optical image OM82 is an optical image of a thick plate crushed into a spherical shape. Furthermore, radiographic image XM81 is a radiographic image of a thin plate crushed into a spherical shape, and radiographic image XM82 is a radiographic image of a thick plate crushed into a spherical shape. It is difficult to distinguish the details of how the fragments OB are crushed simply by comparing the appearance of the optical images (OM81, OM82). However, by comparing the radiographic images XM81 and XM82, it is possible to easily perform image analysis of the crushing of the fragments OB in both cases. If one were to attempt to distinguish by image analysis of the optical images alone, a huge number of optical images taken from various directions would be required. However, by using radiographic images in combination, it is possible to easily perform image analysis of the crushing of the fragments OB.

[0061] Optical image OM83 is an optical image of a thin plate that has been wrinkled without being crushed, and optical image OM84 is an optical image of a thick plate that has been slightly deformed. Furthermore, radiographic image XM83 is a radiographic image of a thin plate that has been wrinkled without being crushed, and radiographic image XM84 is a radiographic image of a thick plate that has been slightly deformed. The same is true for the optical images (OM83, OM84), and it is difficult to determine the details of how the fragments OB have been crushed simply by comparing their appearances. However, by comparing radiographic images XM83 and XM84, it is easy to perform image analysis of how the fragments OB have been crushed in both cases.

[0062] 9(A) and 9(B) are diagrams illustrating the relationship between the distance in the radiation irradiation direction and the thickness of the fragments OB in the irradiation direction, as determined by analyzing a radiographic image. In FIGS. 9(A) and 9(B), arrow directions 901 and 902 indicate the irradiation direction in which radiation is irradiated onto the fragments OB from a direction perpendicular to the paper surface. The thickness along the irradiation direction was measured from the intensity of the radiation that penetrated the fragments OB when irradiated from a direction perpendicular to the paper surface. The graphs in FIGS. 9(A) and 9(B) are diagrams illustrating the relationship between the distance along the irradiation direction and the thickness when radiation is irradiated onto the fragments OB in the optical images (OM91-92) from a direction perpendicular to the paper surface (arrow directions 901 and 902).

[0063] Optical image OM91 in Figure 9(A) is an optical image of a thin plate with almost no deformation, and optical image OM92 in Figure 9(B) is an optical image of a thick plate with almost no deformation. Comparing the appearance of the optical images, they are indistinguishable, but the only difference is the plate thickness. While optical images make it difficult to distinguish between thin and thick plates, using radiographic images makes it easy to distinguish between them. The plate thickness measurement in step S52 can be performed, for example, by determining the distribution of thickness relative to the radiation irradiation direction (distance), as exemplarily illustrated in Figures 9(A) and 9(B).

[0064] In step S61, the processing unit 110 performs a comprehensive judgment process, which comprehensively selects the crushed pieces OB in consideration of the results of the image analysis in S32 and the results of the plate thickness measurement in S52.

[0065] The processing unit 110 may select the fragments OB of the predetermined material using an infrared image, or may select the fragments OB of the predetermined material by combining the results of analysis of a radiological image or an optical image. The processing unit 110 may select the fragments OB of the predetermined material based on the temperature distribution of the fragments OB in the infrared image captured by the infrared imaging unit 12A. The temperature distribution of the fragments OB includes changes in temperature distribution over time after the fragments OB are heated, and the processing unit 110 may select the fragments OB of the predetermined material based on the changes in temperature distribution. Alternatively, the temperature distribution of the fragments OB includes differences in brightness due to temperature changes detected in the infrared image, and the processing unit 110 may select the fragments OB of the predetermined material based on differences in heat conduction patterns that can be calculated from the differences in brightness. In addition, the processing unit 110 may determine that the fragments OB contain a material different from a specified material based on differences in the tendency for the temperature of the fragments OB to rise or the tendency for the temperature of the fragments OB to fall due to differences in the thermal conductivity of the materials, and may exclude fragments OB that contain a different material (e.g., bolts or wiring) from the sorting target (e.g., Figures 5 and 6).

[0066] In addition, the processing unit 110 may determine that an area in the infrared image where the temperature distribution is linear is a joint (e.g., 301, 303) that is brazed using a filler material (e.g., copper) different from a predetermined material (e.g., mild steel) (e.g., Figure 3), and exclude broken fragments OB that include the joint from the selection target.

[0067] Furthermore, the processing unit 110 may determine that crushed fragments OB having a curvature rate equal to or greater than a predetermined value in the temperature distribution of the fragments OB are made of mild steel, which is a predetermined material.

[0068] The processing unit 110 may acquire the outer shape of the fragments OB from a cooled region based on the temperature distribution of the fragments OB (for example, 401 in FIG. 4). The processing unit 110 may acquire the appearance characteristics of the fragments OB based on an optical image, and sort out the fragments OB of a predetermined material using the temperature distribution and the appearance characteristics of the fragments OB. The processing unit 110 may determine that the fragments OB, which tend to show a low temperature distribution in a region determined to be the edge of the plate material constituting the fragments OB based on the appearance characteristics of the fragments OB (for example, 402 in FIG. 4), have a thickness greater than the plate thickness of the predetermined material (for example, 1 mm of mild steel), and exclude the fragments OB from being sorted.

[0069] In addition, the processing unit 110 may obtain the thickness distribution of the fragments OB along the irradiation direction of the radiation irradiated from a direction perpendicular to the paper surface based on the radiation image, for example, as shown in Figures 9(A) and (B), and select fragments OB of a specified material using the temperature distribution of the fragments OB in the infrared image and the thickness distribution of the fragments OB.

[0070] The processing unit 110 outputs the result of the comprehensive judgment process in S61 to the control unit 120. The processing unit 110 transmits to the control unit 120 judgment information indicating the sorting result of the broken pieces OB and position information of the broken pieces OB that were selected.

[0071] In step S72, the control unit 120 sends a movement signal to switch the sorting destination (15A or 15B) of the sorting mechanism 14A based on the position information and determination information of the crushed fragments OB received from the processing unit 110. Based on the movement signal received from the control unit 120, the sorting mechanism 14A adjusts the angle of the sorting panel 14B to switch the sorting destination of the crushed fragments OB transported by the belt 11A to the collection unit 15A or 15B. This allows the crushed fragments OB of a specified material to be collected with high accuracy. According to this embodiment, it is possible to reduce the load in the sorting process and sort crushed fragments of a specified material.

[0072] [Summary of the embodiment] Configuration 1. A sorting device for sorting fragments of a predetermined material from a plurality of fragments obtained by crushing a vehicle body structure, a first optical image capturing unit (12B) for capturing an optical image of the fragments; an infrared image capturing unit (12A) for capturing an infrared image of the fragments; a processing unit (110) that sorts out fragments of the predetermined material based on the optical image and the infrared image; When the processing unit (110) detects a specific area in the infrared image where a temperature contrast with a characteristic shape occurs, it performs image analysis of the optical image limited to an analysis area corresponding to the specific area, thereby sorting out fragments of the specified material.

[0073] Configuration 2: The processing unit (110) detects, as the specific region, a region in the infrared image where the temperature distribution of the fragments differs, The image analysis is performed by limiting the analysis region in the optical image to a region where the temperature distribution tendency is different.

[0074] Configuration 3: The processing unit determines whether the fragments are fragments of the predetermined material based on the color information obtained by the image analysis.

[0075] Configuration 4: The processing unit (110) determines whether the fragments are of the predetermined material based on the shape of the fragments obtained by the image analysis.

[0076] Configuration 5: The apparatus further comprises a second optical image capturing unit (12C) that captures an optical image of the fragments at a different angle of view from that of the first optical image capturing unit (12B).

[0077] Configuration 6. When the processing unit (110) determines that the image analysis of the optical image does not achieve the predetermined accuracy of judgment, it sorts out the fragments of the predetermined material by image analysis using the optical image taken by the first optical image capturing unit and the optical image taken by the second optical image capturing unit.

[0078] Configuration 7. A radiation generating unit (13A) that irradiates the fragments with radiation; The apparatus further includes a detection unit (13B) that detects radiation that has passed through the fragments and obtains a radiographic image based on the detected radiation dose.

[0079] Configuration 8. When the processing unit (110) determines that information on the thickness of the fragments is necessary based on the image analysis, it acquires a distribution of the thickness of the fragments in the irradiation direction of the radiation based on the radiological image; The results of the image analysis and the thickness distribution of the fragments are used to select fragments of the specified material.

[0080] Configuration 9: The processing unit (110) detects, as the specific region, a region in the infrared image where the temperature distribution of the fragments is linear; The image analysis is performed by limiting the analysis region in the optical image to a region where the temperature distribution is linear.

[0081] Configuration 10: The processing unit (110) detects, as the specific region, a region in the infrared image where a fibrous object exists in which the temperature distribution of the fragments differs from that of the surrounding area, The image analysis is performed by limiting the analysis region in the optical image to a region where the fibrous object exists.

[0082] Configuration 11. The processing unit (110) detects, as the specific region, a region in the infrared image where a polygonal object exists in which the temperature distribution of the fragments differs from that of the surrounding area, The image analysis is performed by limiting the analysis area in the optical image to an area where the polygonal object exists.

[0083] Configuration 12: The processing unit (110) uses the image analysis to exclude from the selection targets any fragments that contain a material different from the predetermined material.

[0084] Configuration 13. A sorting method for a sorting device that sorts crushed pieces of a predetermined material from a plurality of crushed pieces obtained by crushing a vehicle body structure, comprising: A step of taking an optical image of the fragments by a first optical image taking unit (12B); A step of taking an infrared image of the crushed pieces by an infrared image taking unit (12A); a processing step in which a processing unit (110) sorts out fragments of the predetermined material based on the optical image and the infrared image; In the processing step, when a specific region in which a temperature contrast having a characteristic shape occurs is detected in the infrared image, image analysis of the optical image is performed in a limited analysis region corresponding to the specific region in the optical image, thereby sorting out fragments of the predetermined material. A sorting method in a sorting device that sorts out fragments of a predetermined material from a plurality of fragments obtained by crushing a vehicle body structure, comprising: A step of taking an optical image of the fragments by a first optical image taking unit; a step of capturing an infrared image of the fragments by an infrared image capturing unit; and a processing step of sorting out fragments of the predetermined material based on the optical image and the infrared image, In the processing process, when a specific area in the infrared image where a temperature contrast with a characteristic shape occurs is detected, image analysis of the optical image is performed limited to an analysis area corresponding to the specific area, thereby sorting out fragments of the specified material.

[0085] According to the first to thirteenth embodiments, it is possible to separate crushed pieces of a predetermined material regardless of the state of deformation of the crushed pieces. According to the configurations of the embodiments, it is possible to separate crushed pieces of high-purity iron scrap from scrap of car body structures. Furthermore, it is possible to separate copper (Cu) and other materials with high thermal conductivity. Until now, it was impossible to produce mild steel sheets for automotive exterior panels (essentially ultra-low carbon steel sheets) from commercial scrap containing large amounts of alloys such as carbon (C) and manganese (Mn). Therefore, the only option was to use raw materials derived from iron ore (reduced iron or blast furnace pig iron), as well as mild steel scrap from steel mills and press waste. According to the present embodiment, it is possible to extract only ultra-low carbon mild steel sheets from crushed pieces of used car bodies. By using these raw materials to produce ultra-low carbon mild steel sheets in an electric furnace or the like, horizontal recycling of steel sheets from car bodies to car bodies becomes possible.

[0086] The invention is not limited to the above-described embodiment, and various modifications and variations are possible within the scope of the gist of the invention. [Explanation of symbols]

[0087] 12A: infrared image capturing unit, 12B: first optical image capturing unit, 12C: second optical image capturing unit, 13A: radiation generating unit (generating unit), 13B: radiation detecting unit (detecting unit), 100: sorting device, 110: processing unit, 120: control unit, 150: machine learning model

Claims

1. A sorting device for sorting crushed pieces of a predetermined material from a plurality of crushed pieces obtained by crushing a vehicle body structure, comprising: a first optical image capturing unit that captures an optical image of the fragments; an infrared image capturing unit that captures an infrared image of the fragments; a processing unit that sorts out fragments of the predetermined material based on the optical image and the infrared image, When the processing unit detects a specific area in the infrared image where a temperature contrast with a characteristic shape occurs, the processing unit performs image analysis of the optical image limited to an analysis area corresponding to the specific area, thereby sorting out fragments of the specified material.

2. The processing unit detects, as the specific region, a region in the infrared image where the temperature distribution of the fragments differs, The sorting device according to claim 1 , wherein the image analysis is performed by limiting the analysis area in the optical image to an area where the temperature distribution tendency is different.

3. The sorting device according to claim 1 , wherein the processing unit determines whether the fragments are of the predetermined material based on color information obtained by the image analysis.

4. The sorting device according to claim 1 , wherein the processing unit determines whether the fragments are of the predetermined material based on the shape of the fragments obtained by the image analysis.

5. The sorting device according to claim 1 , further comprising a second optical image capturing unit that captures an optical image of the fragments at a different angle of view from that of the first optical image capturing unit.

6. The sorting device described in claim 5, wherein when the processing unit determines that the image analysis of the optical image does not achieve the specified judgment accuracy, the processing unit sorts out the fragments of the specified material by image analysis using the optical image taken by the first optical image capturing unit and the optical image taken by the second optical image capturing unit.

7. a radiation generating unit that irradiates the fragments with radiation; The sorting device according to claim 1 , further comprising a detection unit that detects radiation that has passed through the fragments and obtains a radiological image based on the detected radiation dose.

8. When the processing unit determines that information on the thickness of the fragments is necessary based on the image analysis, the processing unit acquires a distribution of thicknesses of the fragments in the irradiation direction of the radiation based on the radiological image; The sorting device according to claim 7, wherein the crushed fragments of the predetermined material are sorted using the results of the image analysis and the thickness distribution of the crushed fragments.

9. The processing unit detects, as the specific region, a region in the infrared image where the temperature distribution of the fragments is linear, The sorting device according to claim 1 , wherein the image analysis is performed by limiting the analysis region in the optical image to a region where the temperature distribution is linear.

10. The processing unit detects, as the specific region, a region in the infrared image where a fibrous object exists in which the temperature distribution of the fragments differs from that of the surrounding area, The sorting device according to claim 1 , wherein the image analysis is performed by limiting the analysis area in the optical image to an area where the fibrous object exists.

11. The processing unit detects, as the specific region, a region in the infrared image where a polygonal object in which the temperature distribution of the fragments differs from that of the surrounding area exists, The sorting device according to claim 1 , wherein the image analysis is performed by limiting the analysis area in the optical image to an area in which the polygonal object exists.

12. The sorting device according to claim 1 , wherein the processing unit excludes, from the sorting targets, fragments containing a material different from the predetermined material through the image analysis.

13. A sorting method for a sorting device that sorts crushed pieces of a predetermined material from a plurality of crushed pieces obtained by crushing a vehicle body structure, comprising: A step of taking an optical image of the fragments by a first optical image taking unit; a step of capturing an infrared image of the fragments by an infrared image capturing unit; a processing step in which a processing unit sorts out fragments of the predetermined material based on the optical image and the infrared image; In the processing step, when a specific area in the infrared image where a temperature contrast with a characteristic shape occurs is detected, the sorting method performs image analysis of the optical image limited to an analysis area corresponding to the specific area in the optical image, thereby sorting out fragments of the specified material.

Images