Scrap grade determination device, scrap grade determination system, scrap grade determination method, machine learning model, and program

Abstract

To provide a scrap grade determination technique that is more convincing to both a seller and a buyer than ever before when determining a scrap grade by segmenting an input image acquired by photographing a scrap group.SOLUTION: A scrap grade determination device includes: an acquisition unit that acquires an image of a scrap group to be determined; and a scrap grade determination unit that determines a scrap grade of a scrap present for each of predetermined image areas acquired by segmenting the image acquired by the acquisition unit. The scrap grade determination unit determines the scrap grade of the scrap present in a first image area by using information on the first image area which is a scrap grade determination target, and information on a second image area surrounding the first image area, in the image acquired by the acquisition unit.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a scrap grade determination device, a scrap grade determination system, a scrap grade determination method, a machine learning model, and a program.

Background Art

[0002] With the growing global interest in CO2 reduction, in the steel industry, the recycling of iron has attracted attention, and an increase in the domestic circulation volume of scrap is expected. Most of the iron scrap is heavy scrap, and for heavy scrap, grades (e.g., HS, H1, H2, ···, etc.) are determined according to thickness, dimensions (width or height × length), and unit weight. For example, heavy scrap with a thickness of 6 mm or more, a width (or height) × length of 500 mm or less × 700 or less, and a unit weight of 600 kg or less is classified into the highest grade HS, and heavy scrap with a thickness of 6 mm or more, a width (or height) × length of 500 mm or less × 1200 or less, and a unit weight of 1000 kg or less is classified into the next highest grade H1. Similarly, heavy scrap is classified into H2, H3, and H4.

[0003] In the circulation of scrap, the transaction unit price (yen / ton) varies according to these grades. In trading, the buyer determines the grade by visually checking the scrap brought in by the seller, and the transaction price is determined. Such a grade determination operation requires skill, and there is also a problem that there are variations in the grade determination by each scrap grade determination facility or each inspector. In response to this problem, Patent Documents 1 to 4 describe technologies for imaging the appearance of scrap as visually inspected by a person with a camera, inputting the captured image into a machine learning model, and automatically determining the scrap grade.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

[0005] [Non-Patent Document 1] Takayuki Okaya, "Research Trends in Deep Learning for Image Recognition - Development of Convolutional Neural Networks and Their Usage Methods -", Journal of the Japanese Society for Artificial Intelligence, Vol. 31, No. 2, pp. 169-179, March 2016 [Summary of the Invention] [Problems to be Solved by the Invention]

[0006] In Patent Documents 1, 3, and 4, one scrap grade is determined for one input image that images a mixed scrap group using a machine learning model. For this reason, it is unclear which scrap in the input image influenced the scrap grade determination, and there was a problem that sufficient acceptability could not be obtained for both the seller and the buyer. On the other hand, in Patent Document 2, one input image is subdivided, and one scrap grade is determined for each of the subdivided images. Therefore, it becomes clear which scrap in the input image influenced the scrap grade determination. However, since the scrap grade is determined independently for each of the subdivided images, there are cases where, for example, adjacent images adjacent to each other in the vertical or horizontal direction are determined to have significantly different scrap grades, and there is still a possibility that sufficient acceptability cannot be obtained for both the seller and the buyer.

[0007] In view of the above problems, one problem of the present disclosure is to provide a scrap grade determination technique that is more acceptable to both sellers and buyers than in the past when determining the scrap grade by subdividing an input image obtained by imaging a scrap group.

Means for Solving the Problems

[0008] One aspect of the present disclosure includes an acquisition unit that acquires an image obtained by imaging a scrap group to be determined, and a scrap grade determination unit that determines the scrap grade of the existing scrap for each predetermined image region obtained by subdividing the image acquired by the acquisition unit. The scrap grade determination unit determines the scrap grade of the scrap existing in the first image region by using the information of the first image region that is the determination target of the scrap grade and the information of the second image region around the first image region in the image acquired by the acquisition unit. The present disclosure relates to a scrap grade determination device.

Effects of the Invention

[0009] According to the present disclosure, when determining the scrap grade by subdividing an input image obtained by imaging a scrap group, it is possible to determine a scrap grade that is more acceptable to both sellers and buyers than in the past.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Mode for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0012] In the following embodiments, a scrap grade determination device that determines the scrap grade of each scrap imaged in the image of a scrap group to be determined is disclosed.

[0013] [Summary of the Present Disclosure] In the scrap grade determination according to an embodiment of the present disclosure, for an image obtained by imaging a group of scraps to be determined, scrap grade determination is performed for each image region or each pixel having a predetermined size, and a scrap grade determination result for each image region or each pixel is output. In the example shown in FIG. 1, for one image obtained by imaging a group of scraps contained in a container 20, the captured image is subdivided into a grid pattern for each image region, and a scrap grade is determined for each image region from image region A1 to AN, and a determination result as shown in the figure is obtained. In this determination result, for example, no scrap is imaged in image regions A1, A2, AN, and the scrap grades of these image regions are determined as "N / A". On the other hand, the scrap in image region AX1 is determined as having a scrap grade of "HS", the scrap in image region AX2 is determined as having a scrap grade of "H4", the scrap in image region AX3 is determined as having a scrap grade of "H2", and the scrap in image region AX4 is determined as having a scrap grade of "H1". Similarly, scrap grades are determined for scraps in other image regions within the image.

[0014] In the illustrated example, the scrap grade is determined for each image region having a predetermined size, but the image region may be each pixel, and the scrap grade of the scrap imaged in each pixel can be determined. Thereby, the scrap grade of each scrap in the group of scraps imaged in one image can be determined.

[0015] [Scrap Grade Determination Model] FIG. 2 is a schematic diagram showing a scrap grade determination device 100 according to an embodiment of the present disclosure. As shown in FIG. 2, the scrap grade determination device 100 acquires a plurality of images #1 to #I obtained by imaging a group of scraps to be determined. For example, the images #1 to #I may be images obtained by imaging a group of scraps contained in the container 20 from different angles and different imaging positions by a camera, or images obtained by moving a group of scraps on the surface of the container 20 by a heavy machine or the like, and thereby imaging a group of scraps inside the container 20 exposed on the surface by the camera. It is preferable from the viewpoint of determining the scrap grade in consideration of not only the group of scraps on the surface of the container 20 but also the group of scraps inside the container 20 to acquire a plurality of images by imaging the group of scraps inside the newly exposed container 20 a plurality of times while moving the group of scraps on the surface.

[0016] When the images #1 to #I are acquired, the scrap grade determination device 100 inputs each of the images #i (i = 1, 2, ···, I) of the images #1 to #I to the scrap grade determination model 50, and acquires a scrap grade determination result for the image #i from the scrap grade determination model 50. As described above with reference to FIG. 1, the scrap grade determination result for the image #i may indicate a scrap grade determination result for each predetermined image region or each pixel of the scrap.

[0017] When the scrap grade determination results for each image are acquired for all of the images #1 to #I, the scrap grade determination device 100 can aggregate the scrap grade determination results for each image of the images #1 to #I and output a scrap grade determination result for the group of scraps contained in the container 20.

[0018] In the illustrated example, the scrap grade determination model 50 is stored in the scrap grade determination device 100. However, the scrap grade determination model 50 according to the present disclosure is not limited thereto and may be provided in an external device such as a cloud server communicatively connected to the scrap grade determination device 100. In this case, the scrap grade determination device 100 may transmit each image #i to the external device and receive, from the external device, the scrap grade determination result for the image #i as the execution result of the scrap grade determination model 50 executed on the image #i by the external device.

[0019] Here, the scrap grade determination device 100 may be realized by a computing device such as a server or a personal computer (PC). For example, it may have a hardware configuration as shown in FIG. 3. That is, the scrap grade determination device 100 includes a storage device 101, a processor 102, an interface device 103, and a communication device 104 that are interconnected via a bus B.

[0020] Programs or instructions for realizing various functions and processes described later in the scrap grade determination device 100 may be downloaded from any external device via a network or the like, or may be provided from a removable storage medium such as a CD-ROM (Compact Disk-Read Only Memory) or a flash memory.

[0021] The storage device 101 is realized by a random access memory, a flash memory, a hard disk drive, etc., and stores files, data, etc. used for the execution of programs or instructions together with the installed programs or instructions. The storage device 101 may include a non-transitory storage medium.

[0022] The processor 102 may be implemented by one or more CPUs (Central Processing Units), GPUs (Graphics Processing Units), processing circuitry, etc. that may be composed of one or more processor cores. According to programs, instructions, data such as parameters necessary to execute the program or instruction stored in the storage device 101, etc., the various functions and processes of the scrap grade determination device 100 described later are executed.

[0023] The interface device 103 realizes the interface between the user of the scrap grade determination device 100 and the user. For example, the user operates a keyboard, mouse, etc. on the GUI (Graphical User Interface) displayed on the display or touch panel, and transmits and receives various information, data, instructions, etc. to and from the scrap grade determination device 100 via the interface device 103.

[0024] The communication device 104 is realized by various communication circuits that execute communication processing with external devices, communication networks such as the Internet, LAN (Local Area Network), etc.

[0025] However, the above-described hardware configuration is merely an example, and the scrap grade determination device 100 according to the present disclosure may be realized by any other appropriate hardware configuration.

[0026] [Scrap Grade Determination Device] Next, the scrap grade determination device 100 according to an embodiment of the present disclosure will be described. FIG. 4 is a block diagram showing the functional configuration of the scrap grade determination device 100 according to an embodiment of the present disclosure.

[0027] As shown in FIG. 4, the scrap grade determination device 100 includes an acquisition unit 110 and a scrap grade determination unit 120. Each functional unit of the acquisition unit 110 and the scrap grade determination unit 120 may be realized by a computer program stored in the storage device 101 of the scrap grade determination device 100 being executed by the processor 102.

[0028] The acquisition unit 110 acquires an image obtained by imaging a group of scraps to be determined. Specifically, the acquisition unit 110 acquires one or more images obtained by imaging a group of scraps to be determined accommodated in a container 20 imaged by a camera or the like. For example, the group of scraps to be determined may be a group of scraps accommodated on the loading platform of a truck that has entered a scrap grade determination facility for selling the scraps. The group of scraps accommodated in the loading platform or the container 20 transferred from the loading platform is photographed by the camera from various angles and shooting positions, or the group of scraps on the surface of the loading platform or the container 20 is moved by a heavy machine or the like, so that the group of scraps inside the loading platform or the container 20 exposed on the surface is photographed by the camera. In order to appropriately evaluate the entire group of scraps, it is preferable that the entire group of scraps in the loading platform or the container 20 is imaged by the camera.

[0029] The scrap grade determination unit 120 determines the scrap grade of the existing scraps for each predetermined image area obtained by subdividing the image acquired by the acquisition unit 110. Here, the scrap grade determination unit 120 uses the information of the image area that is the determination target of the scrap grade and the information of the peripheral image areas around the determination target image area in the image acquired by the acquisition unit 110 to determine the scrap grade of the scraps existing in the determination target image area.

[0030] For example, as shown in FIG. 5, when determining the scrap grade of the image region AX to be determined, the scrap grade determination unit 120 determines the scrap grade of the image region AX in consideration of not only the imaging content of the image region AX but also the imaging content of the peripheral image region of the image region AX. In this way, the scrap grade of the image region AX to be determined is determined based on not only the imaging content of the image region AX but also the imaging content of the peripheral image region, and the scrap grade is determined considering more imaging content. Thereby, it becomes possible to determine the scrap grade of each image region with higher accuracy.

[0031] To achieve this, for example, the concept of the receptive field in a deep learning model may be utilized. The size of the receptive field is a quantity that represents how far away the information of pixels is ultimately used when determining whether a pixel is a feature region (see, for example, Non-Patent Document 1). That is, taking the case where the image region to be determined is each pixel as an example, considering the peripheral image region of the image region to be determined together with the imaging content of the image region to be determined means that the receptive field is larger than 1.

[0032] In one embodiment, the scrap grade determination unit 120 may use a scrap grade determination model 50 trained to receive an image of a scrap group as input and output a scrap grade determination result of the scrap imaged in each image region (for example, an image region of a predetermined size, etc.) or each pixel in the input image, to determine the scrap grade for each image region or each pixel. That is, the scrap grade determination unit 120 determines the scrap grade of the scrap using the pre-created scrap grade determination model 50, and the scrap grade determination model 50 is a machine learning model (for example, a deep learning model, etc.) that takes an image as input and outputs, for each predetermined image region obtained by subdividing the input image, the confidence level indicating which of the predetermined scrap grades the existing scrap corresponds to. However, the scrap grade determination model 50 is not limited to this, and may be any other appropriate model such as a mathematical model or a statistical learning model.

[0033] Specifically, when the scrap grade determination model 50 receives each image as input, it may determine the scrap grade for each predetermined image region or each pixel of the input image. For example, as shown in FIG. 6, when receiving the image #i (i = 1, 2, ···, I) as input, the scrap grade determination model 50 determines the scrap grade of each image region #j of the image #i. For example, when each image #i is composed of (L × M) image regions, the scrap determination model 50 can output (L × M) scrap grade determination results.

[0034] Here, the scrap grade determination result of each image region #j output from the scrap grade determination model 50 can be represented, for example, by the confidence levels of the scrap grades of HS, H1, H2, H3, H4, and N / A. Specifically, the scrap grade determination result is expressed in the vector form of (HS, H1, H2, H3, H4, N / A) = (x1, x2, x3, x4, x5, x6), where x1 + x2 + x3 + x4 + x5 + x6 = 1.0 may be normalized. Then, the scrap grade with the highest confidence level may be determined as the scrap grade of the image region #j, or alternatively, the confidence vector (x1, x2, x3, x4, x5, x6) itself may be determined as the scrap grade of the image region #j.

[0035] Such a scrap grade determination model 50 may be realized by, for example, any type of machine learning model capable of image processing. Specifically, the scrap grade determination model 50 may be generated by supervised learning using a training data set composed of a pair of an image of a scrap group and a scrap grade label for each image region in the image. That is, the image of the scrap group in the training data set is input into the scrap grade determination model to be trained, and the parameters of the scrap grade determination model to be trained are adjusted so as to minimize the error between the output result from the scrap grade determination model to be trained and the scrap grade label for each image region in the image. Then, when the above-described parameter adjustment process is completed for all the training data in the training data set, the finally obtained scrap grade determination model can be used in the scrap grade determination device 100 as the trained scrap grade determination model 50.

[0036] As described above with reference to FIG. 5, when determining the scrap grade of the image region or pixel to be determined, the scrap grade determination unit 120 may determine the scrap grade of the image region or pixel to be determined in consideration of not only the imaging content of the image region or pixel to be determined but also the imaging content of the peripheral image region or pixel of the image region or pixel to be determined. For this reason, for example, the scrap grade determination model 50 may be configured such that the receptive field is larger than 1. Specifically, when receiving an image as an input, the scrap grade determination model 50 may determine the scrap grade of the scrap existing for each image region or pixel obtained by subdividing the image, based on the imaging content of the image region or pixel to be determined and the imaging content of the peripheral image region or pixel of the image region or pixel, together with the imaging content of the image region or pixel to be determined. Such a scrap grade determination model 50 may be realized by, for example, a deep learning model, or may be a mathematical model, a statistical learning model, or the like.

[0037] In one embodiment, the scrap grade determination model 50 may be a deep learning model capable of determining the scrap grade within the first image region of the input image in consideration of one or more second image regions around the first image region in the image. For example, such a scrap grade determination model 50 may be realized by a convolutional neural network. Thereby, the determination of the scrap grade for a certain image region or pixel can be performed not only based on the image region or pixel itself but also in consideration of the scraps in the neighboring image regions or pixels, enabling a more accurate determination of the scrap grade. That is, compared with the prior art in which the image of the scrap group to be determined is subdivided and the scrap grade determination is performed independently for each of the subdivided images, the determination of the scrap grade for a certain image region or pixel is performed not only based on the image region or pixel itself but also in consideration of the scraps in the neighboring image regions or pixels, which can make the scrap grades determined for adjacent images adjacent to each other in the vertical and horizontal directions consistent with each other, and a more accurate determination of the scrap grade can be performed. In order to make the scrap grades determined for adjacent images adjacent to each other in the vertical and horizontal directions consistent with each other, it is desirable that the second image region includes four image regions of the same size as the first image region adjacent to the top, bottom, left, and right of the first image region. That is, it is preferable that the size of the second image region is four times or more the size of the first image region.

[0038] FIG. 7 is a block diagram showing the functional configuration of a scrap grade determination apparatus 100 according to another embodiment of the present disclosure. As shown in FIG. 7, the scrap grade determination apparatus 100 includes an integration unit 130 in addition to the acquisition unit 110 and the scrap grade determination unit 120 described above. The acquisition unit 110 and the scrap grade determination unit 120 are the same as those in the embodiment described above with reference to FIG. 4, and redundant descriptions are omitted.

[0039] The integration unit 130 aggregates the determination results of the scrap grades determined for each image area of the image, and determines the scrap grade for the entire image based on the aggregation result. Specifically, when obtaining the scrap grade determination result of each image area #j of image #i, the integration unit 130 may aggregate the determination results of the scrap grades determined for each image area #j of image #i, and determine the scrap grade for the entire image #i based on the aggregation result. For example, the integration unit 130 arithmetically averages the confidence vectors of the scrap grades determined for each image area #j of image #i by the scrap grade determination unit 120 for all image areas #1 to #J, and calculates the averaged confidence vector. Then, the integration unit 130 may determine the scrap grade with the maximum confidence in the averaged confidence vector as the scrap grade of image #i. Alternatively, the integration unit 130 may determine the averaged confidence vector as the scrap grade of image #i. Thereby, based on the scrap grade determination results for each image area or each pixel, the scrap grade determination result for the entire image can be derived.

[0040] Note that in the process of averaging the scrap grade determination results of each image area described above, "N / A" which is not treated as the normal scrap grade may be excluded. For example, if the scrap grade determination results of each image area #j of image #i are expressed in the vector form of (HS, H1, H2, H3, H4, N / A) = (x1, x2, x3, x4, x5, x6) and are normalized such that x1 + x2 + x3 + x4 + x5 + x6 = 1.0, the image area with x6 = 1.0 is excluded from the averaging process, and for the other image areas, for each xp (p = 1 to 5), after re-normalizing excluding x6 by replacing xp with xp / (x1 + x2 + x3 + x4 + x5), it may be subjected to the averaging process.

[0041] In addition, the integration unit 130 may determine the scrap grade of the entire scrap group from a plurality of images of the scrap group to be determined. That is, the acquisition unit 110 acquires a plurality of images #1 to #I of the scrap group to be determined, and the scrap grade determination unit 120 determines the scrap grade for each image #i (i = 1, 2, ···, I) of the acquired plurality of images #1 to #I using the scrap grade determination model 50. Then, the integration unit 130 may aggregate the determination results of the scrap grades for each image #i across the plurality of images #1 to #I, and determine the scrap grade of the scrap group based on the aggregation result. For example, as described above, when the determination result of the scrap grade of each image #i is obtained as a confidence vector, the integration unit 130 may arithmetically average the confidence vectors of each image #i for all images #1 to #I, and determine the averaged confidence vector as the determination result of the scrap grade of the scrap group to be determined.

[0042] For example, as shown in FIG. 8, as the determination result of the scrap grade for the image #i of the images #1 to #I of the scrap group of the container 20 imaged by the scrap grade determination unit 120, the confidence P that the scrap imaged in the image region #j is the grade class k (for example, HS, H1, H2, H3, H4, N / A, etc.) ijk is obtained. Also, let the weight index determined for each grade class be α k and the weight ratio of each grade class k of the scrap group of the container 20 be R k Then,

Equation

[0043] The weight index is a correction coefficient for considering that even if scrap with a high grade such as HS occupies the same-sized image area, the weight of the corresponding scrap is larger. It can be set in advance as a constant. Of course, without performing this correction, in Equation (1), for all grade classes k, α k = 1, such as equal α k can also be used. This corresponds to performing a simple arithmetic mean process.

[0044] In Equation (1), the confidence vector of each image area #j is used. However, the confidence of 1.0 may be assigned to the grade class with the highest confidence in each image area. This is equivalent to replacing P ijk in Equation (1) with

Number

Number

[0045] According to the scrap grade determination device 100 shown in FIG. 7, the scrap grade determination unit 120 inputs each of the plurality of captured images #1 to #I into the scrap grade determination model 50, and obtains the confidence vector of the scrap grade determination result for each image area or pixel #j of each image #i. Then, the integration unit 130 arithmetic-averages the confidence vectors of each image area or pixel #j of image #i for all image areas or pixels, and obtains the averaged confidence vector as the scrap grade determination result of image #i. Further, the integration unit 130 may execute this process for all images #1 to #I, arithmetic-average the confidence vectors of each image #i for all images #1 to #I, and determine the averaged confidence vector as the scrap grade determination result of the scrap group to be determined accommodated in the loading platform or container 20.

[0046] In this way, it is possible to determine the scrap grade of the entire scrap group to be determined accommodated in the loading platform or the container 20. For example, among the confidence vectors arithmetic-averaged for all the images #1 to #I, the scrap grade with the maximum confidence is determined as the scrap grade of the entire scrap group to be determined, and based on the scrap grade uniformly determined for the entire scrap group, the transaction price of the scrap group may be determined. Alternatively, the confidence vector arithmetic-averaged for all the images #1 to #I may be determined as the scrap grade of the entire scrap group to be determined, and based on the weighting of each confidence of the confidence vector and the unit price of the corresponding scrap grade, the transaction price of the scrap group may be determined.

[0047] Also, in one embodiment, the integration unit 130 may adjust the scrap grade based on the past determination results of the scrap grade. The actual determination of the scrap grade may tend to vary depending on the scrap grade determination facility or the inspector. For example, inspector A may tend to determine a relatively high scrap grade, or the scrap grade determination facility B may tend to determine a relatively low scrap grade. Although it is considered impractical to construct and operate individual scrap grade determination models 50 for each such scrap grade determination facility or inspector, it may be considered necessary to incorporate the determination tendency of such determination entities into the scrap grade determination.

[0048] Therefore, the integration unit 130 may adjust the scrap determination result of the scrap grade determination model 50 for each scrap grade determination facility or inspector. That is, the integration unit 130 may adjust the scrap grade P of the scrap grade determination model 50 as follows. ijk may be adjusted.

Equation

Equation

[0049] Alternatively, the adjustment of the scrap grade is not necessarily limited to the learning of C using a deep learning model or the like. For example, for inspector A who has had relatively high scrap grade determinations in the past, the integration unit 130 converts the confidence vector (x1, x2, x3, x4, x5) of the scrap grade determination result obtained from the scrap grade determination model 50 to (x1 + α1, x2 - α1 + α2, x3 - α2 + α3, x4 - α3 + α4, x5 - α4) to increase the confidence in a higher grade. On the other hand, for scrap grade determination facility B that has had relatively low scrap grade determinations in the past, the integration unit 130 converts the confidence vector (x1, x2, x3, x4, x5) of the scrap grade determination result obtained from the scrap grade determination model 50 to (x1 - α1, x2 + α1 - α2, x3 + α2 - α3, x4 + α3 - α4, x5 + α4) to decrease the confidence in a higher grade. Here, the values of α1, α2, α3, α4 may be set based on the past determination records of inspector A and scrap grade determination facility B.

[0050] [Scrap Grade Determination System] Next, the scrap grade determination system 10 according to various embodiments of the present disclosure will be described. FIG. 9 is a block diagram showing a scrap grade determination system 10 according to an embodiment of the present disclosure. In the embodiment shown in FIG. 9, scrap grade determination devices 100_1, 100_2 utilize a common scrap grade determination model 50. In the illustrated embodiment, two scrap grade determination devices 100_1, 100_2 are included in the scrap grade determination system 10, but the scrap grade determination system 10 according to the present disclosure is not limited thereto and may include three or more scrap grade determination devices 100.

[0051] As shown in FIG. 9, the scrap grade determination device 100_1 includes an acquisition unit 110_1, a scrap grade determination unit 120_1, and an integration unit 130_1, and the scrap grade determination device 100_2 includes an acquisition unit 110_2, a scrap grade determination unit 120_2, and an integration unit 130_2. The acquisition units 110_1 and 110_2, the scrap grade determination units 120_1 and 120_2, and the integration units 130_1 and 130_2 each have the same functions as the acquisition unit 110, the scrap grade determination unit 120, and the integration unit 130 described above, and redundant descriptions are omitted.

[0052] In the present embodiment, the scrap grade determination devices 100_1 and 100_2 use the common scrap grade determination model 50 of the scrap grade determination system 10 to determine the scrap grade of the image to be determined. For example, when the scrap grade determination devices 100_1 and 100_2 are installed in different facilities, etc., the scrap grade determination units 120_1 and 120_2 use the common scrap grade determination model 50 stored in the scrap grade determination devices 100_1 and 100_2 respectively to determine the scrap grade for the acquired scrap image. Then, the integration units 130_1 and 130_2 adjust the scrap grade determined according to the determination tendency in the facility, enabling the scrap grade determination along the determination tendency in each facility. According to the present embodiment, by constructing a single common scrap grade determination model 50 in the scrap grade determination system 10, it is possible to realize the scrap grade determination considering the determination tendency in each facility.

[0053] FIG. 10 is a block diagram showing a scrap grade determination system 10 according to an embodiment of the present disclosure. In the embodiment shown in FIG. 10, the scrap grade determination devices 100_1 and 100_2 use the common scrap grade determination model 50 stored in the scrap grade determination device 100_1.

[0054] As shown in FIG. 10, the scrap grade determination device 100_1 includes an acquisition unit 110_1, a scrap grade determination unit 120_1, and an integration unit 130_1, and the scrap grade determination device 100_2 includes an acquisition unit 110_2 and an integration unit 130_2. The acquisition units 110_1 and 110_2, the scrap grade determination unit 120_1, and the integration units 130_1 and 130_2 each have the same functions as the acquisition unit 110, the scrap grade determination unit 120, and the integration unit 130 described above, and redundant descriptions are omitted.

[0055] In this embodiment, the scrap grade determination device 100_1 uses the scrap grade determination model 50 common to the scrap grade determination system 10 to determine the scrap grade of the image to be determined. On the other hand, the scrap grade determination device 100_2 transmits the scrap image to be determined to the scrap grade determination device 100_1 and obtains the determination result of the scrap grade determination in the scrap grade determination unit 120_1. The scrap grade determination device 100_2 transfers the obtained determination result to the integration unit 130_2, adjusts the determination result based on the determination tendency in the facility, and obtains the final scrap grade determination. According to this embodiment, by constructing a single scrap grade determination model 50 common to the scrap grade determination system 10 and storing it in any one of the scrap grade determination devices 100_1, it is possible to realize a scrap grade determination considering the determination tendency in each facility.

[0056] FIG. 11 is a block diagram showing a scrap grade determination system 10 according to an embodiment of the present disclosure. In the embodiment shown in FIG. 11, the scrap grade determination devices 100_1 and 100_2 use the common scrap grade determination model 50 stored in the server 20.

[0057] As shown in FIG. 11, the scrap grade determination devices 100_1 and 100_2 each have an acquisition unit 110_1, 110_2 and an integration unit 130_1, 130_2, and the server 20 has a scrap grade determination unit 120. The acquisition units 110_1, 110_2, the scrap grade determination unit 120, and the integration units 130_1, 130_2 each have the same functions as the acquisition unit 110, the scrap grade determination unit 120, and the integration unit 130 described above, and redundant descriptions are omitted.

[0058] In the present embodiment, the scrap grade determination devices 100_1 and 100_2 use a scrap grade determination model 50 common to the scrap grade determination system 10 to determine the scrap grade of the image to be determined. Specifically, the scrap grade determination devices 100_1 and 100_2 transmit the scrap image to be determined to the server 20 and obtain the determination result of the scrap grade determination in the scrap grade determination unit 120. The scrap grade determination devices 100_1 and 100_2 pass the obtained determination result to the integration units 130_1 and 130_2, adjust the determination result based on the determination tendency in the facility, and obtain the final scrap grade determination. According to the present embodiment, by constructing a single scrap grade determination model 50 common to the scrap grade determination system 10 and storing it in the server 20, it is possible to realize a scrap grade determination considering the determination tendency in each facility.

[0059] Thus, without preparing individual scrap grade determination models 50 that have learned the determination tendencies of each scrap grade determination facility and each inspector, preparing a common scrap grade determination model 50 for the scrap grade determination system 10 can also have the following effects. For example, by providing the integration unit 130 in the scrap grade determination device 100, it is possible to have the integration unit 130 process the determination tendencies specific to the facility where the scrap grade determination device 100 is installed and the inspector, and the scrap grade determination model 50 can be made common. Patent Documents 1 to 4 do not describe having the function of such an integration unit 130 for scrap grade determination. When performing scrap grade determination considering the determination tendencies specific to a facility or an inspector, it is necessary to construct a deep learning model for each facility or each inspector, and the load of preparing a large amount of training data for generating a deep learning model for each facility or each inspector is high, which has been an obstacle to development. According to the present embodiment, even if there are determination tendencies specific to a facility or an inspector, the common scrap grade determination model 50 can be used. For example, by collecting data in cooperation at each facility and aggregating them, it is possible to prepare training data in an amount required for appropriate training. Also, it becomes possible to directly apply the deep learning model constructed by other facilities or companies to one's own company.

[0060] [Scrap Grade Determination Process] Next, the scrap grade determination process according to an embodiment of the present disclosure will be described. The scrap grade determination process is executed by the above-described scrap grade determination device 100. More specifically, it may be realized by one or more processors 102 of the scrap grade determination device 100 executing one or more programs or instructions stored in one or more storage devices 101. Also, the scrap grade determination device 100 may be realized by a plurality of computers, and the plurality of computers may execute the scrap grade determination process. FIG. 12 is a flowchart showing the scrap grade determination process according to an embodiment of the present disclosure.

[0061] As shown in FIG. 12, in step S101, the scrap grade determination device 100 acquires an image obtained by imaging a group of scraps to be determined. For example, the scrap grade determination device 100 can acquire one or more images showing a group of scraps contained in the container 20 imaged by a camera.

[0062] In step S102, the scrap grade determination device 100 determines the scrap grade of the existing scraps for each predetermined image area obtained by subdividing the acquired image. Here, the scrap grade determination device 100 uses, in the acquired image, information on the image area that is the determination target of the scrap grade and information on the peripheral image area around the determination target image area to determine the scrap grade of the scraps existing in the determination target image area. That is, when determining the scrap grade of the determination target image area or pixel, the scrap grade determination device 100 also considers not only the imaging content of the determination target image area or pixel but also the imaging content of the peripheral image area or pixel of the determination target image area or pixel, and determines the scrap grade of the determination target image area or pixel.

[0063] For example, when determining the scrap grade of the determination target image area or pixel, the scrap grade determination device 100 may also consider not only the imaging content of the determination target image area or pixel but also the imaging content of the peripheral image area or pixel of the determination target image area or pixel, and determine the scrap grade of the determination target image area or pixel. Specifically, the scrap grade determination device 100 may use a scrap grade determination model 50 configured such that the receptive field is larger than 1, and determine the scrap grade of the determination target image area or pixel based on the imaging content of the peripheral image area or pixel of the determination target image area or pixel.

[0064] Such a scrap grade determination model 50 may be implemented by a deep learning model such as a convolutional neural network, or may be any appropriate mathematical model, statistical learning model, etc. Specifically, when receiving an image as input, the scrap grade determination model 50 may determine the scrap grade of the scrap existing in each image region or pixel obtained by subdividing the image, together with the imaging content of the image region or pixel to be determined, based on the imaging content of the image regions or pixels around the image region or pixel. Such a scrap grade determination model 50 may be implemented by a deep learning model, for example, or may be a mathematical model, a statistical learning model, etc.

[0065] In step S103, when obtaining the scrap grade determination result for each image region or pixel in the image input to the scrap grade determination model 50, the scrap grade determination device 100 may aggregate the scrap grade determination results for each image region or pixel and determine the scrap grade determination result for the entire image. For example, when obtaining the scrap grade determination result for each image region #j of image #i, the scrap grade determination device 100 may arithmetically average the confidence vectors of the scrap grades determined for each image region #j of image #i for all image regions #j (j = 1, 2, ···, J) to calculate the averaged confidence vector. Then, the scrap grade determination device 100 may determine the scrap grade with the maximum confidence in the averaged confidence vector as the scrap grade of image #i. Alternatively, the scrap grade determination device 100 may determine the averaged confidence vector as the scrap grade of image #i. Thereby, based on the scrap grade determination results for each image region or each pixel, the scrap grade determination result for the entire image can be derived.

[0066] Furthermore, when a plurality of images of the scrap group to be determined are acquired, the scrap grade determination device 100 may aggregate the scrap grade determination results for the entire each image over the plurality of images, and determine the scrap grade of the scrap group stored in the container 20 based on the aggregation result. Also, the scrap grade determination device 100 may adjust the scrap grade obtained from the scrap grade determination model 50 so as not to cause a significant deviation from the past determination results based on the past determination results of the scrap grade.

[0067] According to the above-described embodiment, the scrap grade determination device 100 uses a machine learning model to determine the scrap grade for each image area or each pixel of each image of the scrap group to be determined, considering the image content of the image area or pixel surrounding the image area or pixel. Thereby, compared with the case where one scrap grade determination result is output for the entire single image to be determined from the machine learning model, the scrap grade of each scrap imaged in each image can be determined in more detail, and it becomes possible to provide a more convincing scrap grade determination result to the trading partner who brought in the scrap.

[0068] As described above, the embodiments of the present disclosure have been described in detail. However, the present disclosure is not limited to the above-described specific embodiments, and various modifications and changes are possible within the scope of the gist of the present disclosure described in the claims.

Description of Reference Numerals

[0069] 10 Scrap grade determination system 20 Server 50 Scrap grade determination model 100 Scrap grade determination device 110 Acquisition unit 120 Scrap grade determination unit 130 Integration unit

Claims

1. An acquisition unit that acquires an image obtained by imaging a scrap group to be determined; A scrap grade determination unit that determines the scrap grade of the existing scrap for each predetermined image area obtained by subdividing the image acquired by the acquisition unit; and The scrap grade determination unit determines the scrap grade of the scrap existing in the first image area using information on the first image area that is the determination target of the scrap grade and information on the second image area around the first image area in the image acquired by the acquisition unit. A scrap grade determination device.

2. The scrap grade determination unit determines the scrap grade of the scrap using a previously created scrap grade determination model, The scrap grade determination model Takes an image as input, For each predetermined image area obtained by subdividing the input image, the confidence indicating which of the predetermined scrap grades the existing scrap corresponds to is output. The scrap grade determination device according to claim 1.

3. The scrap grade determination device according to claim 1, further comprising an integration unit that aggregates the determination results of the scrap grades determined for each image area of the image and determines the scrap grade for the entire image based on the aggregation result.

4. The acquisition unit acquires a plurality of images obtained by imaging the scrap group to be determined, The integration unit determines the scrap grade for each of the acquired plurality of images, aggregates the determination results of the scrap grades for each image over the plurality of images, and determines the scrap grade of the scrap group based on the aggregation result. The scrap grade determination device according to claim 3.

5. The integration unit according to claim 3, wherein the integration unit adjusts the scrap grade based on the past determination results of the scrap grade.

6. The scrap grade determination unit Determines the scrap grade using a scrap grade model common to the scrap grade determination models stored in other scrap grade determination devices. The scrap grade determination device according to claim 1.

7. An acquisition unit that acquires an image obtained by imaging a scrap group to be determined; An image output unit that outputs the image acquired by the acquisition unit to another external device connected via a computer network; For each predetermined image region obtained by subdividing the image, using the information of the first image region that is the determination target of the scrap grade and the information of the second image region around the first image region, a determination result acquisition unit that acquires, from the external device, a determination result in which the scrap grade of the scrap existing in the first image region is determined; An integration unit that aggregates the determination results of the scrap grades determined for each image region of the image and determines the scrap grade for the entire image based on the aggregation result, a scrap grade determination device having the same.

8. The external device is another scrap grade determination device connected via a computer network, or a shared server connected via a computer network together with another scrap grade determination device, the scrap grade determination device according to claim 7.

9. A scrap grade determination system having a first scrap grade determination device and a server connected via a computer network, The first scrap grade determination device, A first acquisition unit that acquires an image of a scrap group to be determined; A first image output unit that outputs the image acquired by the first acquisition unit to the server, The server, A scrap grade determination unit that determines the scrap grade of the existing scrap for each predetermined image region obtained by subdividing the image output from the first scrap grade determination device, The first scrap grade determination device, A first determination result acquisition unit that acquires, from the server, a determination result in which the scrap grade of the scrap existing in the image is determined for each predetermined image region obtained by subdividing the image; A first integration unit that aggregates the determination results of the scrap grades determined for each image region of the image and determines the scrap grade for the entire image based on the aggregation result, a scrap grade determination system having the same.

10. Further having a second scrap grade determination device connected to the server via a computer network, The second scrap grade determination device, A second acquisition unit that acquires an image of a scrap group to be determined; A second image output unit that outputs the image acquired by the second acquisition unit to the server, The scrap grade determination unit of the server, Determines the scrap grade of the existing scrap for each predetermined image region obtained by subdividing the image output from the second scrap grade determination device. The second scrap grade determination device acquires, from the server, a determination result obtained by determining the scrap grade of the existing scrap for each predetermined image region obtained by subdividing the image, i.e., a second determination result acquisition unit; and a second integration unit that aggregates the determination results of the scrap grades determined for each image region of the image and determines the scrap grade for the entire image based on the aggregation result. The scrap grade determination system according to claim 9

11. A scrap grade determination system having a first scrap grade determination device and a second scrap grade determination device connected via a computer network, wherein the first scrap grade determination device has a first acquisition unit that acquires an image obtained by imaging a group of scraps to be determined; and a first image output unit that outputs the image acquired by the first acquisition unit to the second scrap grade determination device. The second scrap grade determination device has a second acquisition unit that acquires an image obtained by imaging a group of scraps to be determined; and a scrap grade determination unit that determines the scrap grade of the existing scrap for each predetermined image region obtained by subdividing the image output from the first scrap grade determination device or the image acquired by the second acquisition unit. The first scrap grade determination device has a first determination result acquisition unit that acquires, from the second scrap grade determination device, a determination result obtained by determining the scrap grade of the existing scrap for each predetermined image region obtained by subdividing the image; and a first integration unit that aggregates the determination results of the scrap grades determined for each image region of the image and determines the scrap grade for the entire image based on the aggregation result. The second scrap grade determination device has a second integration unit that aggregates the determination results of the scrap grades determined by the scrap grade determination unit for each image region of the image and determines the scrap grade for the entire image based on the aggregation result. The scrap grade determination system

12. wherein the scrap grade determination unit determines the scrap grade of the scrap using a scrap grade determination model. The scrap grade determination model is created using (1) a first training dataset composed of an image acquired by the first acquisition unit in the first scrap grade determination device and a pair with a scrap grade label for each image region in the image, and (2) a second training dataset composed of an image acquired by the second acquisition unit in the second scrap grade determination device and a pair with a scrap grade label for each image region in the image. The scrap grade determination system according to claim 10.

13. The scrap grade determination unit In an image output from the first scrap grade determination device or the second scrap grade determination device, using information on a first image region that is the object of scrap grade determination and information on a second image region around the first image region, determines the scrap grade of the scrap present in the first image region. The scrap grade determination system according to claim 11.

14. Obtaining an image of a group of scraps to be determined; For each predetermined image region obtained by subdividing the acquired image, determining the scrap grade of the existing scrap, and using information on a first image region that is the object of scrap grade determination and information on a second image region around the first image region in the acquired image to determine the grade of the scrap present in the first image region; A scrap grade determination method executed by a computer.

15. Obtaining an image of a group of scraps to be determined; For each predetermined image region obtained by subdividing the acquired image, determining the scrap grade of the existing scrap, and using information on a first image region that is the object of scrap grade determination and information on a second image region around the first image region in the acquired image to determine the grade of the scrap present in the first image region; A program for causing a computer to execute.

16. Obtaining an image of a group of scraps to be determined; For each predetermined image region obtained by subdividing the acquired image, determine the scrap grade of the existing scrap. In the acquired image, use the information of the first image region that is the object of scrap grade determination and the information of the second image region around the first image region to determine the grade of the scrap existing in the first image region. A machine learning model that executes The machine learning model is a machine learning model trained by supervised learning using a training data set composed of a pair of an image and a correct label indicating the scrap grade of the scrap imaged within the image region of the image.

17. The machine learning model according to claim 16, wherein the machine learning model is created using (1) a first training data set composed of an image acquired by a first scrap grade determination device and a pair of the scrap grade label of each image region in the image, and (2) a second training data set composed of an image acquired by a second scrap grade determination device and a pair of the scrap grade label of each image region in the image.

Images