Determination device, sorting device, and determination method

Abstract

To provide a determination device with high determination accuracy of a content ratio of an added metal in a broken piece and simple processing involved in learning and determination.SOLUTION: A determination device comprises a camera and a control device. The camera shoots a broken piece of a metal and generates a broken piece image including the broken piece. The control device determines a content ratio of an added metal in the broken piece appearing in the broken piece image generated by the camera on the basis of the broken piece image. The control device includes a determination model constructed by machine-learning a broken piece image for learning and an index showing the content ratio of the added metal in the broken piece appearing in the broken piece image. The control device determines the content ratio of the added metal in the broken piece on the basis of output results obtained by inputting the broken piece image generated by the camera to the determination model.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] This application mainly relates to a determination device for determining the content ratio of added metal in crushed pieces.

Background Art

[0002] Conventionally, for the purpose of reusing crushed pieces, a technique for determining the content ratio of added metal in crushed pieces is known. Patent Document 1 discloses this type of device.

[0003] Patent Document 1 describes that by measuring the three-dimensional shape of the surface of the crushed pieces, a surface differential parameter, which is an index related to the number of times the unevenness of the surface of the crushed pieces changes, is calculated. Patent Document 1 also describes that an identification algorithm is generated by learning parameters such as the surface differential parameter, weight, and volume. Then, using the identification algorithm, it is determined whether the crushed pieces are derived from carbon steel or special steel.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the method described in Patent Document 1, in addition to the process of measuring the three-dimensional shape and calculating the surface differential parameter, it is also necessary to calculate or measure the weight and volume of the crushed pieces. Therefore, the process of generating the identification algorithm and determining the crushed pieces becomes complicated. Also, Patent Document 1 describes that in the method of sorting crushed pieces by the difference in appearance through image processing, the sorting accuracy becomes insufficient because the color tone and shape are similar.

[0006] This application has been made in view of the above circumstances, and its main objective is to provide a determination device that has high accuracy in determining the content ratio of added metals in crushed fragments and has a simple learning and determination process. [Means for solving the problem]

[0007] The problem that this application aims to solve is as described above, and next, the means for solving this problem and their effects will be explained.

[0008] From the perspective of this application, a determination device having the following configuration is provided. That is, the determination device comprises a camera and a control device. The camera photographs metal fragments and generates a fragment image containing the fragments. The control device determines the proportion of added metal in the fragments shown in the fragment image based on the fragment image generated by the camera. The control device has a determination model constructed by machine learning using training fragment images and an index indicating the proportion of added metal in the fragments shown in the fragment image. Based on the output result obtained by inputting the fragment image generated by the camera into the determination model, Without calculating the wrinkle ratio of the fragment from the fragment image, The proportion of added metals in the aforementioned crushed fragments is determined.

[0009] From the perspective of this application, the following determination method is provided. That is, a camera is used to photograph metal fragments and generate a fragment image containing the fragments. Based on the fragment image generated by the camera, a determination is made regarding the content ratio of the added metal in the fragments shown in the fragment image. The determination is made using a determination model constructed by machine learning with training fragment images and an index indicating the content ratio of the added metal in the fragments shown in the fragment image. Based on the output result obtained by inputting the generated fragment image into the determination model, Without calculating the wrinkle ratio of the fragment from the fragment image, The proportion of added metals in the aforementioned crushed fragments is determined. [Effects of the Invention]

[0010] According to this application, it is possible to realize a determination device that has high accuracy in determining the content ratio of added metals in crushed fragments, and in which the learning and determination processes are simple. [Brief explanation of the drawing]

[0011] [Figure 1] A diagram showing the configuration of the sorting device. [Figure 2] A diagram illustrating the process of constructing a decision model. [Figure 3] A diagram showing each layer of the decision model. [Figure 4] A graph showing the frequency distribution of the final feature values. [Figure 5] A graph showing the frequency distribution of the final output values. [Figure 6] A flowchart illustrating the process of determining whether a sample belongs to the low-concentration group or the high-concentration group using a classification model. [Modes for carrying out the invention]

[0012] Next, embodiments of this application will be described with reference to the drawings.

[0013] The sorting device 1 shown in Figure 1 is a device that sorts crushed fragments according to the proportion of added metals. The crushed fragments are obtained, for example, by processing metal for recycling in a crusher. A crusher is located upstream of the sorting device 1, and the crushed fragments are supplied to the sorting device 1. The crusher may be located in a separate factory or other location from the sorting device 1. In this case, the crushed fragments produced by the crusher are transported and supplied to the sorting device 1.

[0014] The technical details described in this embodiment can also be applied to devices that only determine the content ratio of additive metals and do not perform sorting. A device that does not perform sorting is, for example, a device that measures or inspects crushed pieces.

[0015] As shown in FIG. 1, the sorting device 1 includes a conveying device 10, a camera 12, a lighting device 13, a control device 14, and a sorting unit 15. Further, the sorting device 1 includes a determination device 2 that identifies the crushed pieces placed on the conveying device 10. The determination device 2 is composed of a camera 12, a lighting device 13, and a control device 14.

[0016] There is a supply source upstream of the conveying device 10 that sequentially supplies a plurality of crushed pieces. The conveying device 10 conveys the crushed pieces supplied from the supply source to the downstream side. The conveying device 10 in this embodiment is a belt conveyor, but another conveying device can be used as long as it can convey the crushed pieces. Another conveying device is, for example, a roller conveyor.

[0017] The camera 12 is arranged above the conveying device 10. The camera 12 photographs the crushed pieces conveyed by the conveying device and generates an image including the crushed pieces. Hereinafter, an image including the crushed pieces is referred to as a crushed piece image. The camera in this embodiment has a rectangular imaging area. Instead of this, a line camera with a linear imaging area may be used.

[0018] The lighting device 13 irradiates illumination light onto the crushed pieces imaged by the camera 12. By irradiating illumination light onto the crushed pieces, the image of the crushed pieces obtained by the camera 12 can be sharpened. The lighting device 13 is above the conveying device 10 and is arranged at a position downstream of the camera 12 in the conveying direction. The arrangement of the lighting device 13 is an example. For example, the lighting device 13 may be arranged at a position upstream of the camera 12 in the conveying direction.

[0019] The sorting device 1 sorts the crushed pieces into two stages according to the content ratio of the added metal. Hereinafter, the group with the higher content ratio of the added metal is referred to as the high-content group, and the group with the lower content ratio of the added metal is referred to as the low-content group. Note that the grouping is not limited to two stages, and it may be grouped into three or more stages. For example, when the crushed piece is steel and contains a large amount of added metals such as manganese or chromium, it may not be preferable to use it for cast iron as a recycling application. For example, steel with a high content of manganese or chromium may be less likely to precipitate graphite during solidification in casting, or a brittle structure called chill may be generated. Therefore, by sorting the crushed pieces according to the content ratio of the added metal, it is possible to divide them into steel suitable for recycling applications and steel not suitable for recycling applications. The composition of the crushed pieces is not limited to steel, and other alloys may also be used. Other alloys are, for example, iron alloys other than steel or aluminum alloys. The added metal is not limited to manganese or chromium, and for example, nickel may also be used.

[0020] The control device 14 includes an arithmetic device such as an FPGA, an ASIC, or a CPU, and a storage device such as an SSD or a flash memory. The arithmetic device reads and executes the program stored in the storage device to execute various processes related to the sorting device 1. The control device 14 determines, for example, whether the crushed piece belongs to the high-content group or the low-content group based on the image of the crushed piece. The details of this determination will be described later. The control device 14 further controls the conveying device 10 and the sorting unit 15.

[0021] The sorting unit 15 is configured to switch between sending the crushed pieces conveyed by the conveying device 10 to the first recovery unit 21 via the first conveying device 16, or to the second recovery unit 22 via the second conveying device 17. The first recovery unit 21 is a container that mainly recovers crushed pieces of the low-content group. The second recovery unit 22 is a container that mainly recovers crushed pieces of the high-content group. The sorting unit 15 can send the crushed pieces to the first conveying device 16 by injecting compressed air towards them. If the sorting unit 15 does not inject compressed air, the crushed pieces will fall naturally and be sent to the second conveying device 17. The position and timing of the injection of compressed air by the sorting unit are controlled by the control device 14.

[0022] The sorting unit 15 may sort the crushed pieces of the high-concentration group by injecting compressed air. Furthermore, the sorting unit 15 is not limited to a machine that injects compressed air. For example, the sorting unit 15 may be an arm robot. The arm robot picks up the crushed pieces and places them in different locations depending on whether they belong to the high-concentration group or the low-concentration group.

[0023] Next, we will explain the process of classifying the fragments captured in the fragment images into high-content and low-content groups.

[0024] The control device 14 has a judgment model that has been constructed by performing machine learning in advance. As shown in Figure 2, the judgment model is constructed by machine learning using training data and ground truth data. In this embodiment, the training data is images of fragments, and the ground truth data is whether the fragment belongs to the high-content group or the low-content group. This machine learning is considered supervised learning because it presents ground truth data.

[0025] The fragment images used for training data are preferably fragment images generated using the sorting device 1. This allows the shooting conditions to be the same during the construction of the judgment model and during the operation of the sorting device 1, thereby improving the judgment accuracy. However, training data may also be generated in an environment or with a camera different from that of the sorting device 1. In addition, a certain fragment image may be used as training data, and a rotated version of this fragment image may also be used as separate training data. This allows for the preparation of a large amount of training data with simple processing.

[0026] The correct data can be any index indicating the proportion of added metals in the crushed fragments, and is not limited to high-concentration or low-concentration groups. For example, if the crushed fragments are sorted into three or more groups according to the proportion of added metals, it is preferable to also divide the correct data into three or more groups. Alternatively, estimated or measured values ​​of the proportion of added metals may be used as the correct data.

[0027] Since the machine learning in this embodiment is deep learning, there is no need to extract features from the fragment images and input them; only the fragment images themselves need to be input. In order to obtain a valid conclusion using this judgment model, it is assumed that there is a correlation between the fragment images and the content ratio of the added metals. Therefore, the following explains how the content ratio of the added metals can be determined based on the fragment images.

[0028] Metal additives are added to improve the strength of iron alloys and other materials. Therefore, crushed pieces with a high proportion of metal additives are relatively strong and less prone to deformation when processed by a crusher. Conversely, crushed pieces with a low proportion of metal additives are relatively weak and more prone to deformation when processed by a crusher. Specifically, steel plates with a low proportion of metal additives are more easily bent into a rounded shape when fed into a crusher, resulting in wrinkles appearing more easily on the surface of the crushed pieces.

[0029] Therefore, there is a correlation between the wrinkle ratio, which is the proportion of the surface of the fragment that is wrinkled, and the content ratio of the added metal. Specifically, the higher the wrinkle ratio, the lower the content ratio of the added metal. In addition, since wrinkles appear in the image of the fragment, it is possible to determine the content ratio of the added metal based on the image of the fragment.

[0030] However, since the size and surface shape of the fragments vary, and the amount of light they receive is not constant, it is difficult to accurately determine the wrinkle ratio using a predetermined calculation. In this respect, by using the judgment model described above, it is possible to flexibly consider differences in the degree of wrinkles between fragments of the high-content group and fragments of the low-content group, thereby achieving high judgment accuracy.

[0031] Next, the decision model will be described in more detail. As shown in Figure 3, the decision model includes an input layer, a hidden layer, and an output layer. Furthermore, the decision model in this embodiment is constructed using a convolutional neural network. Therefore, the hidden layer includes a convolutional layer, a pooling layer, and a fully connected layer. The decision model is not limited to a convolutional neural network and may be constructed using other neural networks.

[0032] The input layer receives fragmented images. The convolutional layer uses filters to extract specific features from the image, generating an image with those specific features extracted. The type of features extracted varies depending on the filter. The pooling layer compresses the information by downsampling the image with the extracted features. The decision model includes multiple sets of convolutional and pooling layers, and various features are extracted by passing through these layers.

[0033] The fully connected layer receives the output values ​​from each pooling layer. The fully connected layer outputs the final feature value based on the output from each pooling layer. The output layer receives the final feature value. The final feature value includes a first element that quantifies the probability of belonging to the low-content group and a second element that quantifies the probability of belonging to the high-content group.

[0034] Figure 4 shows the frequency distributions of the first and second elements of the final feature value. The upper graph is for the first element of the final feature value, with the horizontal axis representing the value of the first element and the vertical axis representing the frequency. The lower graph is for the second element of the final feature value, with the horizontal axis representing the value of the second element and the vertical axis representing the frequency. The threshold will be discussed later.

[0035] The output layer performs processing using an activation function. The activation function is, for example, the softmax function. This processing using the activation function transforms the final feature values ​​into the final output values. The final output values ​​include a first element indicating the probability of belonging to the low-content group and a second element indicating the probability of belonging to the high-content group.

[0036] Figure 5 shows the frequency distributions of the first and second elements of the final output value. The upper graph is for the first element of the final output value, with the horizontal axis representing the value of the first element and the vertical axis representing the frequency. The lower graph is for the second element of the final output value, with the horizontal axis representing the value of the second element and the vertical axis representing the frequency.

[0037] Here, after sorting device 1 is in operation, there is a need to change the boundary between the high-content and low-content groups of crushed fragments. For example, a change in the boundary may be desired if the type of crushed fragments changes, the destination of the recycled products changes, or if there is a desire to increase the amount of crushed products classified as low-content.

[0038] The determination model of this embodiment can change the boundary between high-concentration and low-concentration groups by setting thresholds for the final feature quantities or final output values ​​in order to calculate these values. However, as shown in Figure 5, the distribution of the final output values ​​is concentrated in a certain interval, so even if the threshold is changed, the actual boundary is hardly changed. In contrast, as shown in Figure 4, the distribution of the final feature quantities is dispersed over a wide range, so the boundary between high-concentration and low-concentration groups can be changed by changing the threshold. The threshold can be changed, for example, by an operator inputting a threshold by operating a control device or a device that can communicate with the control device.

[0039] Therefore, in this embodiment, the final feature values ​​output by the fully connected layer are extracted as a score, and it is determined whether the data belongs to the high-content group or the low-content group based on the score. Either the first element or the second element may be used as the score, but in this embodiment, the first element is used as the score.

[0040] Next, with reference to Figure 6, the processes performed by the control device 14 will be described.

[0041] The control device 14 acquires images of the fragments generated by the camera 12 capturing images of the fragments from the camera 12 (S101). Next, the control device 14 inputs the fragment images into the judgment model and obtains a score indicating the probability of belonging to the low-concentration group (S102). As described above, the score is the first element of the final feature value, and the higher the probability of belonging to the low-concentration group, the larger the score value.

[0042] Next, the control device 14 compares the score with a threshold (S103). If the control device 14 determines that the score is greater than the threshold (S104), it determines that the fragments shown in the fragment image belong to the low-content group (S105). If the control device 14 determines that the score is less than or equal to the threshold (S104), it determines that the fragments shown in the fragment image belong to the high-content group (S106).

[0043] Next, the control device 14 stores the determination result (S107). Then, the control device 14 injects compressed air into the sorting unit 15 at the timing when the crushed pieces determined to be in the low-content group pass through the sorting unit 15. In this way, the crushed pieces can be sorted according to the content ratio of the added metal.

[0044] As described above, the determination device 2 of this embodiment has the following feature 1. That is, the determination device 2 comprises a camera 12 and a control device 14, and performs the following determination method. The camera 12 photographs metal fragments and generates a fragment image containing the fragments. The control device 14 determines the proportion of added metals in the fragments shown in the fragment image based on the fragment image generated by the camera 12. The control device 14 has a determination model constructed by machine learning using training fragment images and an index indicating the proportion of added metals in the fragments shown in the fragment image. The control device 14 determines the proportion of added metals in the fragments based on the output result obtained by inputting the fragment image generated by the camera 12 into the determination model.

[0045] By using a judgment model constructed through machine learning, it is possible to obtain output results that take into account the relationship and trends between the proportion of added metals in the fragments and the fragment images. Therefore, higher judgment accuracy can be achieved compared to cases where no judgment model is used. In addition, by using fragment images, the machine learning and judgment processes can be simplified.

[0046] The determination device 2 of this embodiment has the following feature 2. That is, multiple stages are set according to the content ratio of the added metal in the crushed fragments. The control device 14 determines which stage the crushed fragments shown in the crushed fragment image belong to.

[0047] This allows the crushed fragments to be classified according to the proportion of added metals, so that, for example, different treatments can be applied to the crushed fragments depending on the proportion of added metals.

[0048] The determination device 2 of this embodiment has the following feature 3. That is, the output result output by the determination model is a score indicating the degree of certainty that the fragments shown in the fragment image belong to a stage.

[0049] By using the score, it is possible to distinguish whether a fragment belongs to a particular stage with a high probability or a low probability.

[0050] The determination device 2 of this embodiment has the following feature 4. That is, a threshold is set for each stage. The control device 14 determines that the device belongs to a stage when the score exceeds the threshold.

[0051] By using a threshold, it is easy to determine whether or not a fragment belongs to a certain stage.

[0052] The determination device 2 of this embodiment has the following feature 5. That is, the control device 14 changes the threshold value based on the information input from the operator.

[0053] By changing the threshold, the judgment criteria for each stage can be adjusted, for example, according to the requirements of the location where the judgment device 2 is installed.

[0054] The determination device 2 of this embodiment has the following feature 6. That is, the determination model is constructed of a neural network including at least an input layer, a hidden layer, and an output layer. Processing using an activation function is performed in the output layer. The input value to the activation function is used as the score.

[0055] The input values ​​to the activation function take on a wider range compared to the output values ​​of the activation function, allowing for appropriate classification of fragments according to the threshold.

[0056] The determination device 2 of this embodiment has the following feature 7. That is, the determination model is constructed using a convolutional neural network and includes at least a convolutional layer, a pooling layer, and a fully connected layer. The output value of the fully connected layer is used as the score.

[0057] Since the output values ​​of the fully connected layer take on a wider range compared to the output values ​​of the activation function, it is possible to appropriately classify the fragments according to the threshold.

[0058] Features 1 through 7 described above can be combined as appropriate, as long as no contradictions arise. For example, feature 3 can be combined with at least one of features 1 or 2. Feature 4 can be combined with at least one of features 1 through 3. Feature 5 can be combined with at least one of features 1 through 4. Feature 6 can be combined with at least one of features 1 through 5. Feature 7 can be combined with at least one of features 1 through 6.

[0059] Although preferred embodiments of this application have been described above, the above configuration can be modified as follows, for example.

[0060] The decision model shown in the above embodiment includes an output layer that performs processing using an activation function, but this output layer may be omitted.

[0061] The flowchart shown in the above embodiment is just one example, and some processes may be omitted, some processes may be modified, or new processes may be added. For example, the process may be performed using the probability of belonging to the high-concentration group as the score, rather than the probability of belonging to the low-concentration group. Alternatively, the process of determining whether a substance belongs to the low-concentration group or the high-concentration group may be omitted, and the score may be stored.

[0062] The functions of the elements disclosed herein can be performed using circuits or processing circuits, including general-purpose processors, dedicated processors, integrated circuits, ASICs (Application Specific Integrated Circuits), conventional circuits, and / or combinations thereof, configured or programmed to perform the disclosed functions. A processor is considered a processing circuit or circuit because it includes transistors and other circuits. In this disclosure, a circuit, unit, or means is hardware that performs the enumerated functions, or hardware programmed to perform the enumerated functions. The hardware may be hardware disclosed herein, or other known hardware that is programmed or configured to perform the enumerated functions. If the hardware is a processor, which is considered a type of circuit, then the circuit, means, or unit is a combination of hardware and software, and the software is used to configure the hardware and / or the processor. [Explanation of symbols]

[0063] 1. Sorting device 2 Judgment device 10 Conveying device 12 cameras 13 Lighting fixtures 14 Control device 15. Sorting Department

Claims

1. A camera that photographs metal fragments and generates fragment images containing those fragments, A control device that determines the proportion of added metals in the fragments shown in the fragment image based on the fragment image generated by the camera, Equipped with, The control device has a judgment model constructed by machine learning using a learning fragment image and an index indicating the content ratio of the added metal in the fragment as seen in the fragment image. The control device is a determination device that determines the proportion of added metal in the fragments without calculating the proportion of wrinkles in the fragments from the fragment images, based on the output result obtained by inputting the fragment images generated by the camera into the determination model.

2. A determination device according to claim 1, Multiple stages are set according to the content ratio of the added metal in the crushed fragments. The control device is a determination device that determines which stage the fragments shown in the fragment image belong to.

3. A determination device according to claim 2, A determination device in which the output result output by the determination model is a score indicating the degree of certainty that the fragments captured in the fragment image belong to the stage.

4. A determination device according to claim 3, A threshold is set for the aforementioned stage, The control device is a determination device that determines that the score exceeds the threshold and that the score belongs to the stage.

5. A determination device according to claim 4, The control device is a determination device that changes the threshold based on information input from the operator.

6. A determination device according to claim 4 or 5, The aforementioned decision model is constructed using a neural network that includes at least an input layer, a hidden layer, and an output layer. The output layer performs processing using an activation function. A determination device in which the input value to the activation function is used as the score.

7. A determination device according to claim 4 or 5, The aforementioned decision model is constructed using a convolutional neural network and includes at least a convolutional layer, a pooling layer, and a fully connected layer. A determination device in which the output value of the fully coupled layer is used as the score.

8. The determination device according to claim 1, A sorting unit sorts the crushed pieces based on the determination result of the determination device, A sorting device equipped with the following features.

9. The metal fragments are photographed with a camera, and an image of the fragments containing them is generated. In a determination method for determining the proportion of added metals in the fragments shown in the fragment image, based on the fragment image generated by the camera, The aforementioned determination is performed using a determination model constructed by machine learning with training images of fragments and an index indicating the content ratio of added metals in the fragments shown in the images. A determination method for determining the content ratio of added metals in a fragment, without calculating the wrinkle ratio of the fragment from the fragment image, based on the output result obtained by inputting the generated fragment image into the determination model.

Images