Selecting device and selecting method

Abstract

The invention relates to a picking device and a picking method. The process load of the sorting process is reduced to sort crushed pieces of a predetermined material. A sorting device for sorting a crushed piece of a predetermined material from a plurality of crushed pieces obtained by crushing a vehicle body structure, the sorting device being provided with: a first optical image capturing unit for capturing an optical image of the crushed piece; an infrared image capturing unit that captures an infrared image of the fragmented sheet; and a processing unit that selects a crushed piece of the predetermined material on the basis of the optical image and the infrared image. When a specific region having a characteristic shape and having a temperature contrast is detected in the infrared image, the processing unit performs image analysis on the optical image by being limited to an analysis region corresponding to the specific region in the optical image, thereby selecting fragments of the predetermined material.

Description

Technical Field

[0001] The present invention relates to a picking device and a picking method. Background Art

[0002] In recent years, efforts have been actively underway to significantly reduce waste generation through waste prevention, reduction, recycling, and reuse. To achieve this goal, research and development is being conducted on the recycling of metal materials such as iron and aluminum that make up vehicle bodies.

[0003] Patent Document 1 discloses a sorting device for selecting mild steel based on the shape of crushed pieces. The sorting device disclosed in Patent Document 1 uses surface information, which represents an index value corresponding to the height of the crushed pieces, to identify portions of the crushed piece's surface where the index value difference from the surrounding area exceeds a threshold as wrinkles. The crushed pieces are then sorted based on the wrinkle ratio, which represents the proportion of wrinkled portions on the crushed piece's surface.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent No. 6726753 Summary of the Invention

[0007] Problems to be solved by the invention

[0008] In the sorting device disclosed in Patent Document 1, damaged mild steel and undamaged high-strength steel are sorted as the components to be sorted based on the ratio of wrinkles generated on the surface of the crushed pieces. Since image analysis is required to determine the ratio of wrinkles based on the image of the entire crushed piece, the processing load for the sorting process increases.

[0009] The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide an advantageous technique for reducing the processing load of a sorting process and sorting crushed pieces of a predetermined material.

[0010] Solutions for solving problems

[0011] A selection device according to one aspect of the present invention selects fragments of a predetermined material from a plurality of fragments obtained by crushing a vehicle body structure, the selection device comprising:

[0012] a first optical image capturing unit configured to capture an optical image of the broken pieces;

[0013] an infrared image capturing unit configured to capture an infrared image of the broken pieces; and

[0014] a processing unit that selects the broken pieces of the predetermined material based on the optical image and the infrared image,

[0015] When the processing unit detects a specific area having a characteristic shape and generating a temperature contrast in the infrared image, the processing unit performs image analysis on the optical image limited to an analysis area corresponding to the specific area, thereby selecting the broken pieces of the predetermined material.

[0016] Another aspect of the present invention is a sorting method in a sorting device for selecting fragments of a predetermined material from a plurality of fragments obtained by crushing a vehicle body structure. The sorting method includes:

[0017] a step of capturing an optical image of the broken pieces by a first optical image capturing unit;

[0018] a step of capturing an infrared image of the broken pieces by an infrared image capturing unit; and

[0019] selecting a processing step of crushed pieces of the given material based on the optical image and the infrared image,

[0020] In the processing step, when a specific area having a characteristic shape and generating a temperature contrast is detected in the infrared image, image analysis of the optical image is performed on an analysis area limited to the analysis area corresponding to the specific area, thereby selecting broken pieces of the predetermined material.

[0021] Effects of the Invention

[0022] According to the present invention, it is possible to reduce the load of the sorting process and sort the crushed pieces of a predetermined material. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a diagram showing a configuration example of a sorting device according to an embodiment.

[0024] Figure 2 It is a diagram showing an overview of the processing of the sorting device according to the embodiment.

[0025] Figure 3 1 is a diagram showing a first detection example of a determination area of ​​an infrared image and an analysis area of ​​an optical image.

[0026] Figure 4 1 and 2 are diagrams illustrating a second detection example of a specific area of ​​an infrared image and an analysis area of ​​an optical image.

[0027] Figure 5 3 is a diagram showing a third detection example of the identification area of ​​the infrared image and the analysis area of ​​the optical image.

[0028] Figure 61 is a diagram illustrating a fourth detection example of a specific area of ​​an infrared image and an analysis area of ​​an optical image.

[0029] Figure 7 The diagram illustrates an infrared image and optical images captured from two directions.

[0030] Figure 8 It is a diagram showing a comparative example of an optical image and a radiographic image.

[0031] Figure 9 This is a diagram illustrating the relationship between the distance in the radiation irradiation direction and the thickness of the fragments in the irradiation direction.

[0032] Description of Reference Numerals

[0033] 12A: Infrared image capturing unit; 12B: First optical image capturing unit; 12C: Second optical image capturing unit; 13A: Radiation generating unit (generating unit); 13B: Radiation detecting unit (detecting unit); 100: Selection device; 110: Processing unit; 120: Control unit; 150: Machine learning model. DETAILED DESCRIPTION

[0034] The following embodiments are described in detail with reference to the accompanying drawings. Furthermore, the following embodiments are not intended to limit the scope of the claims, and the present invention does not require the combination of all features described in the embodiments. Any combination of two or more of the multiple features described in the embodiments is also possible. Furthermore, identical or similar structures are denoted by the same reference numerals, and repeated descriptions are omitted.

[0035] (Configuration Example of the Sorting Device 100)

[0036] Figure 1 This figure shows an example configuration of a sorting device 100 according to an embodiment. Sorting device 100 sorts out shredded pieces of a predetermined material from a plurality of shredded pieces (OB1, OB5, etc.) obtained by shredding a vehicle body structure in a shredder (hereinafter also referred to as a crusher), not shown. In the following description, shredded pieces are collectively referred to as shredded pieces OB.

[0037] In this embodiment, the crushed pieces OB are obtained by crushing the vehicle body structure by a crusher and removing non-ferrous metals by magnetic separation. The shapes of the crushed pieces OB can be classified into the following three types based on the thickness and strength.

[0038] (a Crushed into small pieces (for example, mild steel plate with a thickness of less than 1.0 mm, which is actually ultra-low carbon steel for outer plate)

[0039] (b) Crushed into large pieces (e.g., mild steel plates with a thickness of 2.0 mm or more, essentially hot-rolled steel plates used for reinforcing components)

[0040] (c) Flat, unbroken pieces (high-strength steel (hereinafter referred to as high-strength steel) with a thickness of 1.0 mm or more, which are the main frame components)

[0041] Furthermore, as foreign matter, copper wire (Cu wire), brazing (blaze) parts containing Cu, bolts, and other structural steel may be mixed into the crushed pieces OB.

[0042] The sorting device 100 of this embodiment includes, as components for acquiring image information of the broken pieces OB, an infrared image capturing unit 12A (thermal imaging camera) for capturing infrared images, a first optical image capturing unit 12B (first visible light camera) for capturing optical images, a second optical image capturing unit 12C (second visible light camera), and a radiation capturing unit (radiation generating unit 13A, radiation detecting unit 13B). These components are collectively referred to as "image information acquiring units (12A, 12B, 12C, 13A, 13B)."

[0043] The sorting device 100 includes: image information acquisition units (12A, 12B, 13A, 13B) that acquire image information of the broken pieces OB; a processing unit 110 that processes the information acquired by the image information acquisition units; and a control unit 120 that controls the sorting mechanism 14A based on the determination result of the processing unit 110.

[0044] The processing unit 110 of this embodiment can perform inference processing using the trained machine learning model 150. In machine learning, a pre-prepared infrared image is used as input to the learning model, and the selected judgment result is output as output, thereby forming the trained machine learning model 150.

[0045] In machine learning using infrared images, infrared images containing specific areas with characteristic shapes and temperature contrast (areas with different temperature distributions within the characteristic shapes) are added as input data to the learning model. Examples of specific areas include areas with abrupt temperature changes, such as solder joints, end faces of broken pieces, areas with background components distinct from the broken pieces in the infrared image, areas with copper wire (Cu) windings, and areas containing fasteners such as bolts and nuts.

[0046] Alternatively, in machine learning, information on the temperature distribution of the broken pieces OB in the infrared image may be used as an input image (input data). Alternatively, the difference in temperature distribution over time after heating the broken pieces OB may be used as information on the temperature distribution of the broken pieces OB. Alternatively, the difference in brightness resulting from temperature changes detected in the infrared image (the difference in emissivity calculated based on the brightness) may be used as information on the temperature distribution of the broken pieces OB. Alternatively, the difference in heat conduction pattern or heat conductivity calculated from the change in brightness resulting from temperature changes detected in the infrared image may be used as information on the temperature distribution of the broken pieces OB. Alternatively, the difference in reflectivity or brightness information when irradiated with light of a specific wavelength may be used as information on the temperature distribution of the broken pieces OB. Alternatively, machine learning may be performed in advance by combining these data as the input image (input data) to construct a trained machine learning model 150.

[0047] By performing such machine learning, the area to be analyzed can be locally limited, thereby reducing the computational load. Furthermore, the machine learning model 150 may be configured such that during machine learning, an optical image for learning is prepared, machine learning is performed, and a determination result is outputted indicating whether a specific area exists.

[0048] During the inference phase, infrared images of crushed pieces OB conveyed by the conveying mechanism 11, described later, serve as input data for the machine learning model 150. If the infrared image contains a specific region, the processing unit 110 outputs the determination result. On the other hand, if the infrared image does not contain a specific region, inference processing based on the output data of the machine learning model 150 enables selection of crushed pieces of ultra-low carbon steel for shell plate classified as (a) above (hereinafter also referred to as selected crushed pieces).

[0049] The conveying mechanism 11 is configured to transport multiple crushed pieces OB. Typically, a belt conveyor is employed, in which a drive pulley 11B and a driven pulley 11C drive a belt 11A. The drive pulley 11B is connected to a drive source (not shown), such as a motor. The rotational power from the drive source is transmitted to the drive pulley 11B, thereby enabling the conveying mechanism 11 to convey the crushed pieces OB in a predetermined direction, as indicated by the arrow. The belt 11A forms a conveying path for conveying the crushed pieces OB. The various crushed pieces OB1, OB2, and OB3, obtained by shredding the vehicle body structure in a pulverizer (crusher), can be conveyed in the direction indicated by the arrow.

[0050] exist Figure 1, the left side of the paper is set as the upstream of the conveying direction, and the right side of the paper is set as the downstream of the conveying direction. The conveying direction is set as the X direction, the up and down direction intersecting the X direction is set as the Z direction, and the direction intersecting the XZ direction and perpendicular to the paper is set as the Y direction. The broken pieces OB discharged from the crusher are supplied to the upstream side of the conveying mechanism 11, and the conveying mechanism 11 conveys each broken piece OB from upstream to downstream. On the conveying path conveyed by the belt 11A, the broken pieces OB are photographed by the image information acquisition unit (12A, 12B, 12C, 13A, 13B). The image information photographed by the image information acquisition unit (12A, 12B, 12C, 13A, 13B) is input to the processing unit 110. Moreover, the photography performed by the second optical image photographing unit 12C and the radiation photographing unit (13A, 13B) is auxiliary, and photography can be performed according to the photography instructions of the processing unit 110.

[0051] The infrared image capture unit 12A (thermal imaging camera) is configured to capture images of the crushed pieces OB on the belt 11A in the infrared range, thereby detecting the dose of infrared radiation emitted from the crushed pieces OB. The image information (infrared image) captured by the infrared image capture unit 12A is input to the processing unit 110. Based on the image information (infrared image), the processing unit 110 can obtain information on the temperature distribution within the crushed pieces OB. For example, if the infrared image contains a specific region, the processing unit 110 can simply obtain information on the temperature distribution of the specific region extracted based on the output of the machine learning model 150.

[0052] The first optical image capturing unit 12B (visible light camera) is configured to capture images of the broken pieces OB on the belt 11A in the visible light region. Alternatively, the first optical image capturing unit 12B may be a camera composed of, for example, a CCD / CMOS image sensor. The image information (optical image) captured by the first optical image capturing unit 12B is input to the processing unit 110. The processing unit 110 can obtain appearance information of the broken pieces OB based on the image information (optical image). When temperature information of a specific area is obtained in the infrared image, the processing unit 110 can perform image analysis based on appearance information limited to the specific area, rather than the entire optical image.

[0053] Furthermore, the processing unit 110 obtains positional information of the broken pieces OB on the belt 11A based on the image information (optical image) acquired from the first optical image capturing unit 12B. This positional information of the broken pieces OB is correlated with the image information (infrared image) from the infrared image capturing unit 12A, the image information (optical image) from the second optical image capturing unit 12C, and the radiographic image information from the radiographic units (13A, 13B). Furthermore, when the sorting mechanism 14A switches the sorting destination (15A or 15B), the broken pieces OB to be sorted are determined based on this positional information of the broken pieces OB on the belt 11A.

[0054] The second optical image capturing unit 12C is configured to capture auxiliary optical images. The second optical image capturing unit 12C is located downstream of the first optical image capturing unit 12B and is configured to capture the broken pieces OB at a different viewing angle than the first optical image capturing unit 12B.

[0055] If the processing unit 110 determines that the accuracy of the optical image analysis needs to be further improved, the processing unit 110 outputs a capture instruction signal to the second optical image capturing unit 12C. The second optical image capturing unit 12C is configured to capture an optical image of the broken fragment OB upon receipt of the capture instruction signal. The captured image information (optical image) is input to the processing unit 110. By using multiple optical images at different viewing angles, the processing unit 110 can perform more accurate image analysis.

[0056] and, Figure 1 , two examples of the first optical image capturing unit 12B and the second optical image capturing unit 12C are shown, but the present invention is not limited to this example. Two or more optical image capturing units may be provided. Furthermore, the second optical image capturing unit 12C is not limited to being positioned above the belt 11A. For example, the second optical image capturing unit 12C may be positioned laterally (perpendicular to the paper) of the belt 11A.

[0057] The radiation imaging unit includes a radiation generating unit 13A that irradiates the broken pieces OB with radiation, and a radiation detecting unit 13B that detects the radiation that has passed through the broken pieces OB.

[0058] The radiation generator 13A starts irradiating radiation in response to an irradiation start command from the processing unit 110. The radiation is typically X-rays, but may also be gamma rays or the like. The radiation irradiated from the radiation generator 13A passes through the fragments OB and enters the radiation detector 13B. Furthermore, the radiation generator 13A stops irradiating radiation in response to an irradiation stop command from the processing unit 110.

[0059] As an example configuration, the radiation detection unit 13B may include a radiation detection panel having a two-dimensional pixel array of pixels that detect radiation; a conversion circuit that acquires charge corresponding to the irradiation dose; and an image generation unit that generates a radiographic image based on the charge information. Furthermore, a line sensor having a one-dimensional array of pixels may be used instead of a radiation detection panel having a two-dimensional pixel array. The image generation unit generates a radiographic image based on the charge information corresponding to the radiation incident on the radiation detection panel and transmits the image to the processing unit 110.

[0060] The radiation dose transmitted through the broken pieces OB varies depending on the thickness of the material forming the broken pieces OB and the state of the broken pieces (e.g., whether they are broken into spherical shapes or not broken but wrinkled). This difference in dose may cause shading in the radiographic image showing the interior of the broken pieces OB. The processing unit 110 can determine the thickness of the broken pieces OB or the distribution of the thickness within the broken pieces OB (in the cross-sectional direction) based on the radiographic image.

[0061] (Structure of Processing Unit 110)

[0062] The processing unit 110 may also be composed of a dedicated circuit such as a PLD (Programmable Logic Device) such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). Alternatively, the processing unit 110 may be composed of a combination of a general-purpose processing circuit such as a processor and a storage circuit such as a memory. In this case, the general-purpose processing circuit may execute a program stored in the storage circuit to implement the functions of the processing unit 110.

[0063] The processing unit 110 can perform material selection based on inference processing using deep learning, neural networks, and other techniques. The processing unit 110 uses the image (infrared image) acquired by the infrared image capture unit 12A as input and applies the trained machine learning model 150 to select a predetermined material from the crushed pieces OB. While various materials can be selected, steel plates (ultra-low carbon mild steel plates) with a thickness of less than a predetermined thickness (e.g., less than 1 mm) can also be selected.

[0064] The processing unit 110 transmits determination information indicating the material selection result and position information of the crushed pieces OB to be selected to the control unit 120. Based on the position information and determination information of the crushed pieces OB received from the processing unit 110, the control unit 120 transmits an activation signal to switch the selection destination (15A or 15B) of the selection mechanism 14A.

[0065] The sorting mechanism 14A adjusts the angle of the sorting panel 14B based on the movement signal received from the control unit 120, and switches the sorting destination of the crushed pieces OB conveyed by the belt 11A to the recovery unit 15A or 15B. Figure 1 In the example shown, the state in which the crushed pieces OB4 selected as the predetermined material (ultra-low carbon mild steel plate) are selected and recovered in the recovery unit 15B is shown. Furthermore, the state in which the crushed pieces OB5 of a material other than the predetermined material are selected and recovered in the recovery unit 15A is shown.

[0066] Figure 1 While the illustrated configuration of the sorting mechanism 14A shows an example in which the sorting mechanism 14A changes the angle of the sorting panel 14B based on a motion signal, other mechanisms that achieve equivalent functions may also be used. For example, an air blow system or a robotic arm system may also be used. Recycling units 15A and 15B typically utilize boxes, trays, or the like. Figure 1 In the illustrated configuration example, the number of the recovery modules 15A and 15B is set to two, but the number may be three or more.

[0067] (Overview of Processing of the Selection Device 100)

[0068] Figure 2 This figure schematically illustrates the processing of the sorting device 100 according to the embodiment. In step S21, the first optical image capturing unit 12B captures an image of the broken pieces OB on the belt 11A. The image information captured by the first optical image capturing unit 12B is input to the processing unit 110. Based on the image information acquired from the first optical image capturing unit 12B, the processing unit 110 determines the positional information of the broken pieces OB on the belt 11A (S71).

[0069] In step S31 , the infrared image capturing unit 12A captures an image of the broken pieces OB on the belt 11A. Image information captured by the infrared image capturing unit 12A is input to the processing unit 110 .

[0070] In step S32, the processing unit 110 selects broken pieces of a specific material based on the optical image and the infrared image. If the processing unit 110 detects a specific region with a characteristic shape and temperature contrast in the infrared image, it performs image analysis on the optical image, limiting the analysis area corresponding to the specific region, to thereby select broken pieces of the specific material. The processing unit 110 detects a region in the infrared image where the temperature distribution of the broken pieces differs as the specific region, and performs image analysis on the optical image, limiting the analysis area to the region with the temperature distribution differing. Alternatively, the processing unit 110 determines whether the broken pieces are of the specific material based on color information obtained through image analysis. Alternatively, the processing unit 110 determines whether the broken pieces are of the specific material based on the shape of the broken pieces obtained through image analysis.

[0071] In the inference process using infrared images, the image information (infrared image) obtained from the infrared image capture unit 12A is input into the trained machine learning model 150. This allows the processing unit 110 to obtain the selection results of the crushed pieces OB as the output of the machine learning model 150 as judgment information. The time from the crusher processing to the infrared image capture by the infrared image capture unit 12A can also be used as the elapsed time after heating. When the crusher crushes the vehicle body structure while continuously capturing infrared images, the crushed pieces reach a high temperature due to process heating, so heating of the crushed pieces is unnecessary. Alternatively, when selecting crushed pieces that have been temporarily cooled to room temperature, reheating can be performed using methods other than infrared or high frequency.

[0072] Alternatively, the processing unit 110 may classify the shape of the broken fragments OB based on inference processing based on the infrared image and the appearance characteristics of the broken fragments OB based on the optical image. Specifically, the processing unit 110 uses the trained machine learning model 150 to determine whether a specific region within the broken fragment OB is detected based on the infrared image acquired by the infrared image capture unit 12A. If the specific region is not detected in the infrared image, the broken fragment OB can be determined using inference processing based on the entire infrared image.

[0073] Alternatively, if a specific area is detected in the infrared image, the processing unit 110 can obtain information such as the temperature distribution within the specific area and perform inference processing limited to the specific area. In this case, the processing unit 110 can combine inference processing of the specific area in the infrared image with image analysis of the specific area in the optical image to determine the presence of broken fragments OB. When performing image analysis on the specific area in the optical image, the processing unit 110 can perform image analysis on the specific area in the optical image using the optical image captured in step S21. If temperature information for the specific area is obtained in the infrared image, the processing unit 110 can perform image analysis limited to the specific area, rather than performing image analysis on the entire optical image.

[0074] Figure 3 : is a diagram showing a first detection example of a determination area of ​​an infrared image and an analysis area of ​​an optical image. Figure 3 In the first detection example, the processing unit 110 detects the area where the temperature distribution of the broken pieces in the infrared image is linear as the determination area, and performs image analysis limited to the area where the temperature distribution is linear as the analysis area in the optical image. Figure 3 (A) is a diagram showing a detection example of a brazing portion 301 included in the crushed piece OB and a background portion 302 on the belt 11A on which the crushed piece OB is placed as a detection example of a specific area (a linear temperature rapid change portion). Figure 3 The soldered portion 301 shown in (A) is a portion soldered together using, for example, a Cu-based brazing filler metal (solder), and is distributed linearly in the crushed piece OB. In the background portion 302, a region with low temperature distribution is also distributed linearly.

[0075] Figure 3 (B) is a diagram showing the analysis area limited to the determined area (the soldering portion 301 and the background portion 302) in the optical image captured in S21. The analysis area 303 is the analysis area corresponding to the determined area (the soldering portion 301) in the infrared image. The analysis area 304 is the analysis area corresponding to the determined area (the background portion 302) in the infrared image. The processing unit 110, through image analysis of the analysis area 303, detects that the area with low temperature distribution is the color of copper (Cu), and determines that the analysis area 303 is a soldered portion formed by soldering using a Cu-based solder (solder). In addition, the processing unit 110, through image analysis of the analysis area 304, determines that the area with low temperature distribution is an area with nothing in the optical image, and determines that the analysis area 304 is a reflection of an area unrelated to the broken piece OB.

[0076] Figure 4 : is a diagram showing a second detection example of a determination area of ​​an infrared image and an analysis area of ​​an optical image. Figure 4 In the second detection example, the processing unit 110 also detects the area where the temperature distribution of the broken pieces in the infrared image is linear as the determination area, and performs image analysis limited to the area where the temperature distribution is linear as the analysis area in the optical image. Figure 4 (A) is a diagram showing a detection example of the end surface 401 of the crushed piece OB as a detection example of the specific area (linear temperature rapid change portion). Figure 4 In the end face 401 shown in (A), similarly to the brazed portion 301, the region with low temperature distribution is distributed linearly.

[0077] Figure 4 (B) shows the analysis region limited to the identified area in the optical image captured in S21. Analysis region 402 corresponds to the identified area (end surface 401) in the infrared image. Based on the shape and color information obtained from image analysis of analysis region 402, processing unit 110 determines that the area with the low temperature distribution is not copper (Cu) but the end surface of the crushed piece OB.

[0078] Figure 5 3 is a diagram showing a third detection example of the identification area of ​​the infrared image and the analysis area of ​​the optical image. Figure 5 In the third detection example, the processing unit 110 detects the area where the temperature distribution of the broken pieces in the infrared image is different from that of the surrounding area and where the fibrous object exists as a determined area, and performs image analysis limited to the area where the fibrous object exists as the analysis area in the optical image. Figure 5 (A) is a diagram showing an example of detecting a fibrous object 501 having a lower temperature than the surrounding temperature as an example of detecting a specific area. Figure 5 The fibrous object 501 shown in (A) shows a state in which copper wire (Cu wire) is wound. Figure 5 (B) shows the analysis area limited to the specific region (fibrous object 501) in the optical image captured in S21. Analysis region 502 corresponds to the specific region (fibrous object 501) in the infrared image. Processing unit 110 analyzes the image of analysis region 502 and, based on the color information obtained through image analysis, determines that the specific region (fibrous object 501) is not resin fiber but copper wire (Cu wire).

[0079] Figure 6 : is a diagram showing a fourth detection example of a determination area of ​​an infrared image and an analysis area of ​​an optical image. Figure 6In the fourth detection example, the processing unit 110 detects the area where the temperature distribution of the broken pieces in the infrared image is different from that of the surrounding area and where the polygonal object exists as a determined area, and performs image analysis limited to the area where the polygonal object exists as the analysis area in the optical image. Figure 6 (A) is a diagram showing a detection example of polygonal objects 601 and 602 (for example, hexagonal objects) having a temperature distribution tendency different from that of the surrounding temperature as a detection example of a specific area. Figure 6 (B) shows the analysis area limited to the specific region (hexagonal objects 601 and 602) in the optical image captured in S21. Analysis areas 603 and 604 correspond to the specific regions (hexagonal objects 601 and 602) in the infrared image, respectively. Processing unit 110 analyzes the images of analysis areas 603 and 604 and determines that the specific regions (hexagonal objects 601 and 602) are bolts. Fragments OB, including foreign matter such as bolts and copper wire, are removed from the fragments to be sorted.

[0080] When a specific area is detected in the infrared image, image analysis is performed only on that specific area in the optical image. This locally limits the analysis area, reduces the computational load, and improves determination accuracy. Furthermore, even if there is dirt or residual coating on the surface of the crushed pieces OB, these are not affected, saving computer resources and improving determination accuracy.

[0081] In the image analysis of step S32 , if it is determined that predetermined determination accuracy cannot be achieved by analyzing only the optical image captured by the first optical image capturing unit 12B, the processing unit 110 outputs a capturing instruction signal to the second optical image capturing unit 12C.

[0082] In step S41, the second optical image capturing unit 12C captures an optical image of the broken pieces OB based on the received capture instruction signal. The position information of the broken pieces OB to be captured on the belt 11A is determined, and the broken pieces OB to be captured are the same broken pieces (for example, Figure 1 The second optical image capturing unit 12C captures the broken pieces OB at a different viewing angle from that of the first optical image capturing unit 12B, and transmits the captured image information (optical image) to the processing unit 110.

[0083] The processing unit 110 may classify the shapes of the broken pieces OB based on inference processing based on infrared images or on features of the appearance of the broken pieces OB based on a plurality of optical images having different viewing angles.

[0084] Figure 7 The diagram illustrates an infrared image and optical images captured from two directions. Figure 7 701 of (B) is an infrared image 701 captured by the infrared image capturing unit 12A. Figure 7 702 of (A) is an optical image 702 captured by the first optical image capturing unit 12B. Figure 7 The optical image 703 of (C) is taken by the second optical image taking unit 12C. The direction of the arrow symbol shown in the conveying direction 704 is Figure 1 In the conveying direction of the belt 11A, the crushed pieces OB are sequentially photographed as an optical image 702, an infrared image 701, and an optical image 703 on the path where they are conveyed from the upstream side to the downstream side by the belt 11A.

[0085] Infrared image 701 contains randomly occurring high-temperature areas, and inference processing based on infrared image 701 may make it impossible to distinguish whether these areas represent the shape of a broken piece or residual paint. The optical image captured by first optical image capture unit 12B is an image of one side of broken piece OB. If processing unit 110 determines that combining the analysis results of infrared image 701 and optical image 702 does not achieve the desired accuracy, it outputs a capture instruction signal to second optical image capture unit 12C to capture optical image 703 from a different side. Image analysis of optical images 702 and 703 allows image information to be obtained from optical images taken from different directions for areas identified as high-temperature areas in infrared image 701.

[0086] The processing unit 110 classifies the shape of the broken pieces OB based on the inference processing based on the infrared image 701 and the features of the appearance of the broken pieces OB based on the optical images 702 and 703, thereby being able to determine whether the broken pieces OB are Figure 7 The broken piece OB shows the shape of the broken piece OB, and the high temperature portion is the inside of the broken broken piece OB. In this case, the processing unit 110 determines that the broken piece OB is a soft iron with a thin plate thickness (a broken piece to be selected).

[0087] In the image analysis process of step S32, for example, Figure 4Thus, if the end face 401 of the broken piece OB is visible as the analysis region 402 corresponding to the specified region (end face 401), detailed determination based on the plate thickness is possible, and thus the processing unit 110 proceeds to step S51. Specifically, if image analysis of the optical image determines that information on the thickness of the broken piece is necessary to select broken pieces of a given material, the processing unit 110 proceeds to step S51. Furthermore, image analysis of the optical image is not limited to analysis of the optical image captured by the first optical image capturing unit 12B; if image analysis of the optical image captured by the second optical image capturing unit 12C determines that information on the thickness of the broken piece is necessary, the processing proceeds to step S51.

[0088] In step S51 , the processing unit 110 outputs an irradiation start command to the radiation generating unit 13A to start radiation imaging. The radiation detecting unit 13B generates a radiographic image of the fragmented pieces OB and transmits the image to the processing unit 110 .

[0089] In step S52, the processing unit 110 measures the thickness of the crushed piece OB based on the radiographic image. Here, as a specific example of the radiographic image, Figure 8 The figures show a comparison example of optical images (OM81-84) and radiographic images (XM81-84). Optical image OM81 is an optical image of a thin plate broken into a spherical shape, and optical image OM82 is an optical image of a thick plate broken into a spherical shape. Furthermore, radiographic image XM81 is a radiographic image of a thin plate broken into a spherical shape, and radiographic image XM82 is a radiographic image of a thick plate broken into a spherical shape. Comparing only the appearance of the optical images (OM81 and OM82) makes it difficult to discern the details of the damage pattern of the broken pieces OB. However, comparing radiographic images XM81 and XM82 makes it easy to perform image analysis of the damage pattern of both broken pieces OB. If one attempts to discern the damage solely through image analysis of the optical images, a large number of optical images taken from various directions are required. However, by using radiographic images in combination, image analysis of the damage pattern of the broken pieces OB can be easily performed.

[0090] Optical image OM83 is an optical image of a thin plate that is not damaged but deformed into a wrinkled shape, while optical image OM84 is an optical image of a thick plate that is slightly deformed. Furthermore, radiographic image XM83 is a radiographic image of a thin plate that is not damaged but deformed into a wrinkled shape, while radiographic image XM84 is a radiographic image of a thick plate that is slightly deformed. Similarly, in the optical images (OM83 and OM84), it is difficult to discern the details of the damage pattern of the broken pieces OB simply by comparing their appearance. However, by comparing radiographic images XM83 and XM84, it is easy to perform image analysis of the damage patterns of both broken pieces OB.

[0091] Figure 9 (A) and (B) are diagrams illustrating the relationship between the distance in the radiation irradiation direction and the thickness of the fragment OB in the irradiation direction, obtained by analyzing the radiation image. Figure 9 In (A) and (B), the arrow symbol directions 901 and 902 indicate the irradiation directions of the broken pieces OB from a direction perpendicular to the paper surface. The plate thickness along the irradiation direction is measured based on the intensity of the radiation that passes through the broken pieces OB when irradiated from a direction perpendicular to the paper surface. Figure 9 The graphs (A) and (B) illustrate the relationship between the distance and thickness along the irradiation direction when the broken pieces OB of the optical image (OM91-92) are irradiated with radiation from a direction perpendicular to the paper surface (directions of arrows 901 and 902).

[0092] Figure 9 The optical image OM91 of (A) is an optical image of a thin plate with almost no deformation, Figure 9 The optical image OM92 in (B) is an optical image of a thick plate with almost no deformation. Although no difference can be found in the appearance of the optical image, the difference is only in the thickness of the plate. In this way, although it is difficult to distinguish in an optical image, using a radiographic image, it is easy to distinguish between thin plates and thick plates. For example, Figure 9 As exemplified in (A) and (B) of FIG. 5 , the plate thickness measurement in step S52 can be performed by obtaining the distribution of the thickness with respect to the irradiation direction (distance) of the radiation.

[0093] In step S61, the processing unit 110 performs comprehensive determination processing. This processing comprehensively selects the crushed pieces OB in consideration of the result of the image analysis in S32 and the result of the plate thickness measurement in S52.

[0094] The processing unit 110 may use infrared images to select broken pieces of a given material, or may use analysis results of radiographic images and optical images to select broken pieces OB of a given material. Alternatively, the processing unit 110 may select broken pieces OB of a given material based on the temperature distribution of the broken pieces OB in the infrared image captured by the infrared imaging unit 12A. Alternatively, the temperature distribution of the broken pieces OB may include a change in the temperature distribution over time after the broken pieces OB are heated, and the processing unit 110 may select broken pieces OB of a given material based on the change in the temperature distribution. Alternatively, the temperature distribution of the broken pieces OB may include a difference in brightness due to temperature change detected in the infrared image, and the processing unit 110 may select broken pieces OB of a given material based on a difference in a heat conduction pattern that can be calculated from the difference in brightness. Alternatively, the processing unit 110 may determine that the broken pieces OB include a material different from the given material based on a difference in the tendency of the broken pieces OB to increase in temperature or a difference in the tendency of the broken pieces OB to decrease in temperature due to differences in the thermal conductivity of the materials, and may exclude broken pieces OB containing a different material (e.g., bolts or wiring) from the selection targets (e.g., Figure 5 、 Figure 6 ).

[0095] Alternatively, the processing unit 110 may determine that a region in which the temperature distribution is linear in the infrared image is a joint portion (e.g., 301, 303) formed by brazing using a solder (e.g., copper) different from the predetermined material (e.g., mild steel) (e.g., Figure 3 ), remove the broken pieces OB including the joint parts from the selected objects.

[0096] Alternatively, the processing unit 110 may determine that a broken broken piece OB having a curvature greater than or equal to a predetermined value in the temperature distribution of the broken piece OB is made of a predetermined material, namely, soft steel.

[0097] Alternatively, the processing unit 110 may obtain the outer shape of the broken piece OB (for example, Figure 4 Alternatively, the processing unit 110 may obtain the appearance features of the broken pieces OB based on the optical image, and select the broken pieces OB of a given material using the temperature distribution of the broken pieces OB and the appearance features of the broken pieces OB. Alternatively, the processing unit 110 may select the broken pieces OB of a given material in an area (for example, Figure 4 In step 402), the crushed piece OB showing a tendency of low temperature distribution is determined to be thicker than the plate thickness of a predetermined material (for example, 1 mm plate thickness of mild steel), and the crushed piece OB is removed from the selected objects.

[0098] Alternatively, the processing unit 110 may be configured to process the radiographic image, for example, Figure 9 As shown in (A) and (B), the distribution of the thickness of the broken pieces OB along the irradiation direction of the radiation irradiated from a direction perpendicular to the paper surface is obtained, and the temperature distribution of the broken pieces OB and the thickness distribution of the broken pieces OB in the infrared image are used to select the broken pieces OB of a predetermined material.

[0099] The processing unit 110 outputs the result of the comprehensive determination process in S61 to the control unit 120. The processing unit 110 transmits to the control unit 120 determination information indicating the result of sorting of the crushed pieces OB and position information of the crushed pieces OB to be sorted.

[0100] In step S72, the control unit 120 transmits a motion signal to switch the sorting destination (15A or 15B) of the sorting mechanism 14A based on the positional information and determination information of the crushed pieces OB received from the processing unit 110. Based on the motion signal received from the control unit 120, the sorting mechanism 14A adjusts the angle of the sorting panel 14B, switching the sorting destination of the crushed pieces OB conveyed by the belt 11A to the recovery unit 15A or 15B. This allows crushed pieces OB of a given material to be accurately recovered. According to this embodiment, the load of the sorting process can be reduced while sorting crushed pieces of a given material.

[0101] [Summary of Implementation Methods]

[0102] Structure 1. A selection device for selecting broken pieces of a predetermined material from a plurality of broken pieces obtained by breaking a vehicle body structure, the selection device comprising: a first optical image capturing unit 12B for capturing an optical image of the broken pieces; an infrared image capturing unit 12A for capturing an infrared image of the broken pieces; and a processing unit 110 for selecting the broken pieces of the predetermined material based on the optical image and the infrared image. When the processing unit 110 detects a specific area having a characteristic shape and generating a temperature contrast in the infrared image, the processing unit 110 performs image analysis on the optical image limited to an analysis area corresponding to the specific area in the optical image, thereby selecting the broken pieces of the predetermined material.

[0103] In configuration 2, the processing unit 110 detects a region in the infrared image where the temperature distribution of the broken pieces has a different tendency as the determination region, and performs the image analysis limited to the region in the optical image where the temperature distribution has a different tendency.

[0104] Configuration 3: The processing unit determines whether the fragments are broken pieces of the predetermined material based on the color information obtained by the image analysis.

[0105] Configuration 4: The processing unit 110 determines whether the fragments are fragments of the predetermined material based on the shapes of the fragments obtained by the image analysis.

[0106] Configuration 5: The optical image capturing unit 12C is further provided, and the optical image capturing unit 12C captures an optical image of the broken fragments at a viewing angle different from that of the first optical image capturing unit 12B.

[0107] Structure 6: When the processing unit 110 determines that a predetermined determination accuracy cannot be obtained in the image analysis of the optical image, the processing unit 110 uses the optical image captured by the first optical image capturing unit and the optical image captured by the second optical image capturing unit to perform image analysis, thereby selecting broken pieces of the predetermined material.

[0108] Configuration 7 further includes: a generator 13A for irradiating the fragments with radiation; and a detector 13B for detecting the radiation that has passed through the fragments and acquiring a radiographic image based on the detected radiation dose.

[0109] Structure 8: When the processing unit 110 determines that information on the thickness of the broken pieces is required through the image analysis, it obtains the distribution of the thickness of the broken pieces in the irradiation direction of the radiation based on the radiation image, and uses the results of the image analysis and the distribution of the thickness of the broken pieces to select broken pieces of the predetermined material.

[0110] In structure 9, the processing unit 110 detects the area where the temperature distribution of the broken pieces in the infrared image is linear as the determination area, and performs the image analysis limited to the area where the temperature distribution is linear as the analysis area in the optical image.

[0111] Structure 10, the processing unit 110 detects the area where the temperature distribution of the broken pieces is different from the surrounding area and where the fibrous object exists as the determined area, and limits the image analysis to the area where the fibrous object exists as the analysis area in the optical image.

[0112] Structure 11, the processing unit 110 detects the area where the temperature distribution of the broken pieces is different from the surrounding area and where the polygonal object exists as the determined area, and performs the image analysis limited to the area where the polygonal object exists as the analysis area in the optical image.

[0113] The structure 12 and the processing unit 110 remove broken pieces containing materials different from the predetermined material from the selected objects through the image analysis.

[0114] Structure 13. A selection method is a selection method in a selection device, wherein the selection device selects broken pieces of a predetermined material from a plurality of broken pieces obtained by crushing a vehicle body structure, the selection method comprising: a step of capturing an optical image of the broken pieces by a first optical image capturing unit 12B; a step of capturing an infrared image of the broken pieces by an infrared image capturing unit 12A; and a processing step of selecting the broken pieces of the predetermined material based on the optical image and the infrared image by a processing unit 110, wherein, in the processing step, when a specific area having a characteristic shape and generating a temperature contrast is detected in the infrared image, image analysis of the optical image is performed on the optical image limited to an analysis area corresponding to the specific area in the optical image, thereby selecting the broken pieces of the predetermined material.

[0115] According to structures 1 to 13, it is possible to select shredded pieces of a predetermined material regardless of the deformation state of the shredded pieces. According to the structure of the embodiment, it is possible to select shredded pieces of high-purity iron scrap from scrap of vehicle body structures. It is also possible to select copper (Cu) and the like with high thermal conductivity. Until now, it has not been possible to manufacture soft steel plates for automobile exterior panels (essentially ultra-low carbon steel plates) from urban waste containing a large amount of alloys such as carbon (C) and manganese (Mn). The main raw materials can only be raw materials from iron ore (reduced iron or blast furnace pig iron), and waste materials in ironworks and soft steel scraps caused by stamping chips can be used. According to this embodiment, it is possible to extract only ultra-low carbon soft steel plates from shredded pieces of used automobile bodies, use them as raw materials, and manufacture ultra-low carbon soft steel plates through electric furnaces, etc., thereby realizing parallel recycling of steel plates from automobile bodies to automobile bodies.

[0116] The present invention is not limited to the above-described embodiment, and various modifications and changes can be made within the scope of the gist of the invention.

Claims

1. A sorting device for selecting fragments of a predetermined material from a plurality of fragments obtained by crushing a vehicle body structure, the sorting device comprising: a first optical image capturing unit configured to capture an optical image of the broken pieces; an infrared image capturing unit configured to capture an infrared image of the broken pieces; and a processing unit that selects the broken pieces of the predetermined material based on the optical image and the infrared image, When the processing unit detects a specific area having a characteristic shape and generating a temperature contrast in the infrared image, the processing unit performs image analysis on the optical image limited to an analysis area corresponding to the specific area, thereby selecting the broken pieces of the predetermined material.

2. The selection device according to claim 1, characterized in that The processing unit detects a region in the infrared image where the tendency of the temperature distribution of the broken pieces is different as the specific region, The image analysis is performed by limiting the area where the tendency of the temperature distribution is different as the analysis area in the optical image.

3. The selection device according to claim 1, characterized in that The processing unit determines whether the fragments are broken pieces of the predetermined material based on the color information obtained by the image analysis.

4. The selection device according to claim 1, characterized in that The processing unit determines whether the crushed pieces are crushed pieces of the predetermined material based on the shapes of the crushed pieces obtained by the image analysis.

5. The selection device according to claim 1, characterized in that The optical image capturing unit further includes a second optical image capturing unit configured to capture an optical image of the broken fragments at a viewing angle different from that of the first optical image capturing unit.

6. The selection device according to claim 5, characterized in that When the processing unit determines that a predetermined determination accuracy cannot be achieved in the image analysis of the optical image, the processing unit performs image analysis using the optical image captured by the first optical image capturing unit and the optical image captured by the second optical image capturing unit to thereby select broken pieces of the predetermined material.

7. The selection device according to claim 1, characterized in that Also features: a generating unit configured to irradiate radiation toward the fragments; and The detection unit detects the radiation transmitted through the fragments and acquires a radiographic image based on the detected dose of the radiation.

8. The selection device according to claim 7, characterized in that When the processing unit determines that information on the thickness of the fragments is necessary through the image analysis, the processing unit acquires a distribution of the thickness of the fragments in the irradiation direction of the radiation based on the radiographic image. The broken pieces of the given material are selected using the result of the image analysis and the distribution of the thickness of the broken pieces.

9. The selection device according to claim 1, characterized in that The processing unit detects an area in the infrared image where the temperature distribution of the broken pieces is linear as the specific area. The image analysis is performed by limiting the area where the temperature distribution is linear as the analysis area in the optical image.

10. The selection device according to claim 1, characterized in that The processing unit detects an area in the infrared image where the temperature distribution of the broken pieces is different from that of the surroundings and where a fibrous object exists as the specific area. The image analysis is performed by limiting the area where the fibrous object exists as the analysis area in the optical image.

11. The selection device according to claim 1, characterized in that: The processing unit detects an area in the infrared image where the temperature distribution of the broken pieces is different from that of the surroundings and where a polygonal object exists as the specific area. The image analysis is performed by limiting the area where the polygonal object exists as the analysis area in the optical image.

12. The selection device according to claim 1, characterized in that The processing unit removes broken pieces containing a material different from the predetermined material from the selected objects through the image analysis.

13. A method for selecting, in a selecting device, fragments of a predetermined material from a plurality of fragments obtained by crushing a vehicle body structure, the method comprising: a step of capturing an optical image of the broken pieces by a first optical image capturing unit; a step of capturing an infrared image of the broken pieces by an infrared image capturing unit; as well as The processing unit selects the broken pieces of the predetermined material based on the optical image and the infrared image. In the processing step, when a specific area having a characteristic shape and generating a temperature contrast is detected in the infrared image, image analysis of the optical image is performed on an analysis area limited to the analysis area corresponding to the specific area, thereby selecting broken pieces of the predetermined material.

Images