Ratio determination system, ratio determination method, ratio determination program, recording medium, and storage place selection method

Abstract

Provided are a ratio determination system, a ratio determination method, a ratio determination program, a recording medium, and a storage place selection method capable of improving determination accuracy of a presence ratio for each type of target object in a target object aggregate in which a plurality of types of target objects coexist. Specifically, a ratio determination system (1) comprises: an imaging device (2) that captures an image including a target object aggregate in which a plurality of types of target objects coexist; and a computation device (3) that computes a presence ratio of each type of target object in the target object aggregate included in the image. The computation device (3) includes a ratio calculation unit (3B) that calculates the presence ratio of each type of target object by using a ratio determination model (5A), which is a trained model generated in advance by machine learning using the image including the target object aggregate as input data and the presence ratio of each type of target object in the target object aggregate as teacher data.

Description

Ratio determination system, ratio determination method, ratio determination program, recording medium, and storage location selection method

[0001] The present invention relates to a ratio determination system, a ratio determination method, a ratio determination program, a recording medium, and a storage location selection method.

[0002] Conventionally, from the perspective of effective utilization of resources, waste such as scrap (iron scrap) has been reused as recyclable resources. However, in order to reuse scrap, it is necessary to determine the grade of the scrap. Therefore, in order to determine the grade of scrap, a system that calculates the ratio of the grades included in a set of scrap using a pre-trained model has already been disclosed (for example, Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2021-157725

[0004] However, in the method of Patent Document 1, an image obtained by photographing a set of scrap is divided into a plurality of image pieces, and for each of the divided image pieces, a pre-trained model generated by machine learning is used to uniquely determine the grade of the corresponding scrap. In this method, since the scrap having high saliency in the image has a great influence on the determination of the grade, there is a problem that the result often differs from the determination of the grade of the scrap actually observed by the inspector.

[0005] The present invention has been made in view of such circumstances, and an object thereof is to provide a ratio determination system, a ratio determination method, a ratio determination program, a recording medium, and a storage location selection method capable of calculating the existence ratio of each type of target object in a target object aggregate.

[0006] (1) According to one aspect of the present invention, a ratio determination system is provided, comprising: an imaging device that captures an image containing a collection of multiple types of objects; and a calculation device that calculates the proportion of each type of object in the collection of objects contained in the image, wherein the calculation device has a ratio calculation unit that calculates the proportion of each type of object using a pre-trained model generated by machine learning with the image containing the collection of objects as input data and the proportion of each type of object in the collection of objects as training data.

[0007] (2) In the proportion determination system of (1) above, the imaging device has a position information acquisition unit that acquires position information of the object collection, and the position information acquisition unit acquires position information of the object collection from the image containing the object collection using a pre-trained model generated by machine learning with the image containing the object collection as input data and the position information of the object collection as training data.

[0008] (3) In the proportion determination system of (1) above, the imaging device has a position information acquisition unit that acquires position information of a transport device that transports the object assembly, and the position information acquisition unit acquires the position information of the transport device from the image containing the transport device using a pre-trained model generated by machine learning with the image containing the transport device as input data and the position information of the transport device as training data.

[0009] (4) The proportion determination system described in (2) above is characterized in that the imaging device has an imaging location setting unit that sets one or more imaging locations based on the acquired position information of the object collection.

[0010] (5) In the ratio determination system described in (3) above, the photographing device is characterized in that it has a photographing location setting unit that sets one or more photographing locations based on the acquired location information of the transport device.

[0011] (6) In the ratio determination system of (4) or (5) above, the shooting device is characterized in that it has a camera control unit that controls the shooting direction and magnification of the camera based on the shooting location set by the shooting location setting unit.

[0012] (7) In the ratio determination system described in (6) above, the ratio calculation unit is characterized in that it calculates the proportion of each type of target object for each of the shooting locations.

[0013] (8) In the ratio determination system of (7) above, the calculation device is characterized in that it has an overall calculation unit that calculates the proportion of each type of target object in the collection of target objects based on the proportion of each type of target object calculated for each shooting location.

[0014] (9) In the proportion determination system of (1) to (8) above, the calculation device has a calculation unit that stores the unit price for each type of object, and the calculation unit calculates the price of the object collection based on the proportion of existence and the unit price.

[0015] (10) In the proportion determination system of (1) to (9) above, the calculation device has a calculation unit that stores the component composition of each type of target object, and the calculation unit calculates the component composition of the target object aggregate based on the proportion of existence and the component composition.

[0016] (11) In the proportion determination system of (1) to (10) above, the calculation device has a calculation unit that stores judgment criteria for classifying the object collection into pass and fail, and the calculation unit determines whether the object collection is pass or fail based on the existence proportion and the judgment criteria.

[0017] (12) In the proportion determination system of (1) to (11) above, the target object is characterized in that it is scrap.

[0018] (13) According to one aspect of the present invention, a ratio determination method is provided, comprising the steps of: taking an image of an object collection containing a mixture of multiple types of object collections; and calculating the proportion of each type of object present in the object collection contained in the image, wherein the method is characterized in that the proportion of each type of object is calculated using a pre-trained model generated by machine learning, with the image containing the object collection as input data and the proportion of each type of object present in the object collection as training data.

[0019] (14) According to one aspect of the present invention, a program is provided that causes a computer to take an image containing a collection of objects of multiple types, and to calculate the proportion of each type of object in the collection of objects contained in the image, wherein the proportion of each type of object is calculated using a pre-trained model generated by machine learning, with the image containing the collection of objects as input data and the proportion of each type of object in the collection of objects as training data.

[0020] (15) According to one aspect of the present invention, there is a computer-readable recording medium that stores a program that causes a computer to take an image containing a collection of objects of multiple types, and to calculate the proportion of each type of object in the collection of objects contained in the image, wherein the proportion of each type of object is calculated using a pre-trained model generated by machine learning, with the image containing the collection of objects as input data and the proportion of each type of object in the collection of objects as training data.

[0021] (16) According to one aspect of the present invention, a method for selecting a storage location is provided, characterized in that the type of object in the object collection is determined based on the proportion of each type of object in the object collection calculated using the proportion determination system described in (1) to (12) above, and the object collection is transported to a storage location determined for each type of object according to the determination.

[0022] According to the proportion determination system, proportion determination method, proportion determination program, and recording medium of the present invention, the proportion of each type of object in an object collection can be calculated. Furthermore, according to the placement selection method, the placement of the object collection can be determined more quickly based on the proportion of each type of object in the object collection.

[0023] This figure shows an overview of the proportion determination system. This figure shows an example of training images for the proportion determination model. This figure shows an example of training data for the proportion determination model. This figure shows an example of training images for the aggregate recognition model. This flowchart shows an example of the procedure for the proportion determination method. This figure shows an example of implementation by the proportion determination system. This figure shows a comparison of acquired images.

[0024] Embodiments of the present invention will be described below with reference to the drawings. Each drawing is schematic and may differ from the actual one. Furthermore, the following embodiments are illustrative of an apparatus or method for embodying the technical idea of ​​this disclosure and do not limit the configuration to those described below. That is, the technical idea of ​​the present invention can be modified in various ways within the technical scope described in the claims.

[0025] [First Embodiment] The proportion determination system 1 in this embodiment is used, for example, at a scrap receiving site in a steel mill using an electric furnace or blast furnace. Specifically, the proportion determination system 1 is a system for photographing a scrap assembly 11 with a camera and determining the proportion of each grade of scrap in the scrap assembly 11 based on the captured image. In this embodiment, the proportion of existence refers to the proportion by weight. The following describes the case in this embodiment where the target object in the object assembly to be proportion-determined is scrap, but the target object is not limited to scrap.

[0026] Scrap aggregate 11 contains a mixture of scrap of various grades. Scrap grades are determined according to type, dimensions (specifically, thickness, width (height), length), and unit weight (for example, HS to H4). Currently, the unified standards for iron scrap acceptance revised in 2008 by the Japan Iron Source Association are widely used.

[0027] [Configuration] Figure 1 shows an overview of the ratio determination system 1. The ratio determination system 1 comprises a shooting device 2, a calculation device 3, a learning device 4, and a storage device 5. These devices can communicate with each other via a communication network such as a LAN. Furthermore, all or part of these devices may be configured as an integrated device.

[0028] The imaging device 2 has a computer including a CPU, RAM, ROM, non-volatile memory, and input / output interfaces, and the CPU performs information processing according to a program loaded from ROM or non-volatile memory into RAM. The program may be supplied via an information recording medium. Examples of information recording media include optical discs such as CD-ROMs and DVD-ROMs, magnetic media such as magnetic disks, magnetic tapes, and floppy disks, and semiconductor memory such as flash memory, SSDs, and ROMs. It may also be supplied via a communication network such as the Internet or a LAN. Furthermore, the program may be placed on a server and provided in a form that can be downloaded to the device via a communication network such as the Internet or a LAN.

[0029] The imaging device 2 includes an imaging unit 2A, a position information acquisition unit 2B, a shooting location setting unit 2C, and a camera control unit 2D.

[0030] The imaging unit 2A processes images of the scrap pile 11 and its surroundings using optical imaging means such as a camera. The position information acquisition unit 2B acquires position information of the scrap pile 11 from the images captured by the imaging unit 2A using the scrap pile recognition model 5B. The shooting location setting unit 2C sets one or more shooting locations so that the scrap pile 11 and the shooting locations overlap, based on the acquired position information of the scrap pile 11. The camera control unit 2D controls, for example, the horizontal operation, vertical operation, and telephoto and wide-angle of the PTZ camera to capture the set shooting locations.

[0031] The arithmetic unit 3 is composed of a computer including a CPU, RAM, ROM, non-volatile memory, and input / output interfaces, and the CPU performs information processing according to a program loaded from ROM or non-volatile memory into RAM. The arithmetic unit 3 has an image acquisition unit 3A, a ratio calculation unit 3B, an overall calculation unit 3C, and a calculation unit 3D.

[0032] The image acquisition unit 3A acquires images captured by the imaging device 2. The acquired images do not necessarily have to be used directly in the ratio calculation unit 3B; they may be cropped or otherwise processed as appropriate.

[0033] The ratio calculation unit 3B uses the ratio determination model 5A, described later, to calculate the proportion of each grade of scrap in the scrap aggregate 11 included in the image acquired by the image acquisition unit 3A. For example, if the scrap is divided into grades A, B, and C, the proportion of each grade in the scrap aggregate 11 is calculated from the acquired image as follows: 40% for grade A, 30% for grade B, and 30% for grade C.

[0034] The overall calculation unit 3C calculates the proportion of each grade of scrap in the scrap assembly 11 as a whole, based on the proportion of each grade of scrap in the scrap assembly 11 calculated by the proportion calculation unit 3B. One calculation method is, for example, averaging. The following are numerical examples for the sake of explanation and correspond to examples of training data. Referring to the values ​​in Figure 3, the average proportion of each scrap grade is "33% for grade A, 37% for grade B, and 30% for grade C". Therefore, the proportion of each grade in the scrap assembly 11 as a whole is "33% for grade A, 37% for grade B, and 30% for grade C".

[0035] The known technology shown in Figure 3 refers to the scrap grade determination in the scrap assembly 11 using the technology disclosed in Patent Document 1. When using the known technology, only grade A is present in image 11A, only grade B is present in image 11B, and only grade A is present in image 11C. Therefore, the proportion of each grade in the scrap assembly 11 is "67% for grade A, 33% for grade B, and 0% for grade C." From the above, it can be seen that the proportion determination system 1 in this embodiment can accurately calculate the proportion of each grade of scrap in the scrap assembly 11 compared to the known technology.

[0036] The calculation unit 3D stores the unit price for each grade of scrap, the composition of each scrap, and the criteria for sorting the scrap assemblies 11 into acceptable and unacceptable categories. Therefore, the calculation unit 3D can calculate the price of the scrap assemblies 11 based on the proportion of each grade of scrap and the unit price for each grade of scrap. Furthermore, it can calculate the composition of the scrap assemblies 11 based on the proportion of each grade of scrap and the composition of each grade of scrap. In addition, it can determine whether the scrap assemblies 11 are acceptable or unacceptable based on the proportion of each grade of scrap and the criteria for sorting the scrap assemblies 11 into acceptable and unacceptable categories.

[0037] The imaging unit 2A, position information acquisition unit 2B, shooting location setting unit 2C, camera control unit 2D, image acquisition unit 3A, ratio calculation unit 3B, overall calculation unit 3C, and calculation unit 3D may be configured as software modules that realize their functions by executing the program of the present invention on a processor.

[0038] The learning device 4 is composed of a computer including a CPU, RAM, ROM, non-volatile memory, and input / output interfaces, and the CPU performs information processing according to a program loaded from ROM or non-volatile memory into RAM. The learning device 4 generates a trained model, which will be described later. The trained models in this embodiment are a ratio determination model 5A and a group recognition model 5B. Each trained model is used in the ratio determination system 1. The ratio determination model 5A is used in the ratio calculation unit 3B of the arithmetic unit 3. The group recognition model 5B is used in the position information acquisition unit 2B of the imaging device 2. The learning device 4 includes an image acquisition unit 4A, an image processing unit 4B, and a machine learning unit 4C.

[0039] The image acquisition unit 4A acquires images to be used for machine learning. The type of image is not particularly limited, but it may be an image taken by the shooting device 2 or an image acquired from an external source as a training image. The image processing unit 4B performs image processing such as object recognition and object detection on the image acquired by the image acquisition unit 4A. At that time, the image processing unit 4B may divide the acquired image into two or more parts. The machine learning unit 4C generates the ratio determination model 5A used in the ratio calculation unit 3B and the aggregate recognition model 5B used in the location information acquisition unit 2B from the acquired image.

[0040] In this embodiment, a pre-trained model generated by machine learning using an image containing the scrap aggregate 11 as input data and the proportion of each grade of scrap in the scrap aggregate 11 as training data is designated as the proportion determination model 5A. On the other hand, a pre-trained model generated by machine learning using an image containing the scrap aggregate 11 as input data and the positional information of the scrap aggregate 11 as training data is designated as the aggregate recognition model 5B.

[0041] The storage device 5 is a recording medium. Examples of recording media include optical discs such as CD-ROMs and DVD-ROMs, magnetic media such as magnetic disks, magnetic tapes, and floppy disks, and semiconductor memories such as flash memories, SSDs, and ROMs. The storage device 5 stores the ratio determination model 5A and the aggregate recognition model 5B generated by the learning device 4.

[0042] [Method for Generating Learned Model] Hereinafter, a method for generating a learned model will be described. First, the image acquisition unit 4A acquires an image used for machine learning. The type of the image is not particularly limited, and it may be an image captured by the imaging device 2 or an image acquired from the outside as a learning image. Next, the image processing unit 4B performs image processing such as object recognition and object detection on the image acquired by the image acquisition unit 4A. At that time, the image processing unit 4B may divide the acquired image into two or more parts. Finally, the machine learning unit 4C performs machine learning using the image processed by the image processing unit 4B as input data and the information included in the image processed by the image processing unit 4B as teacher data. In the present embodiment, the machine learning unit 4C generates the ratio determination model 5A used by the ratio calculation unit 3B and the aggregate recognition model 5B used by the position information acquisition unit 2B.

[0043] [Ratio Determination Model] FIG. 2 is a diagram showing an example of an image used for machine learning as input data when generating the ratio determination model 5A. As an example, an image including the scrap aggregate 11 is divided vertically into three parts, and the divided images 11A, 11B, and 11C are used as input data respectively.

[0044] FIG. 3 is a diagram showing an example of numerical values used for machine learning as teacher data when generating the ratio determination model 5A. Note that this numerical example is a value obtained by an actual inspector observing the scrap and calculating the existence ratio for each grade of the scrap. In the present embodiment, the existence ratio for each grade in each of the images 11A, 11B, and 11C obtained by dividing the image including the scrap aggregate 11 into three parts is used as teacher data. The grades of the scrap are set to three levels: grade A, grade B, and grade C, and the existence ratio for each grade of the scrap is shown.

[0045] As Pattern 1, for each image containing the three-part scrap collection 11, the proportion of each grade is set as training data. In the example shown in Figure 3, the proportion of each grade of scrap in image 11A is "50% for grade A, 20% for grade B, and 30% for grade C". Similarly, the proportion of each grade of scrap in image 11B is "10% for grade A, 60% for grade B, and 30% for grade C", and the proportion of each grade of scrap in image 11C is "40% for grade A, 30% for grade B, and 30% for grade C".

[0046] Therefore, using image 11A as input data, machine learning is performed with the training data that the proportion of scraps of each grade contained in image 11A is "50% for grade A, 20% for grade B, and 30% for grade C". Furthermore, using image 11B as input data, machine learning is performed with the training data that the proportion of scraps of each grade contained in image 11B is "10% for grade A, 60% for grade B, and 30% for grade C". In addition, using image 11C as input data, machine learning is performed with the training data that the proportion of scraps of each grade contained in image 11C is "40% for grade A, 30% for grade B, and 30% for grade C".

[0047] Furthermore, the above training data does not represent grade determination based solely on the surface of the scrap aggregate 11. The above training data takes into account the weight, thickness, and depth of the scrap. In addition, the grade determination is based not only on the surface of the scrap aggregate 11 but also on the hidden scrap, and the data represents actual grade determinations made by inspectors.

[0048] As Pattern 2, it is assumed that the existing ratio for each grade in the scrap aggregate 11 is previously known to be "Grade A is 33%, Grade B is 37%, and Grade C is 30%". In such a case, even if the existing ratio for each grade in each of the images 11A, 11B, and 11C is "Grade A is 33%, Grade B is 37%, and Grade C is 30%", it may be acceptable. Therefore, using the images 11A, 11B, and 11C as input data respectively, with the existing ratio for each grade in each image being "Grade A is 33%, Grade B is 37%, and Grade C is 30%" as teacher data, machine learning may be performed.

[0049] Pattern 1 can calculate the existing ratio for each grade of the scrap with high precision, but it is difficult to collect teacher data. Therefore, in Pattern 1, it often takes time and cost to generate the ratio determination model 5A. On the other hand, although the accuracy of Pattern 2 for the existing ratio for each grade of the scrap is lower than that of Pattern 1, the collection of teacher data is relatively easy. Therefore, in Pattern 2, the time and cost for generating the ratio determination model 5A can be suppressed.

[0050] From the above, a ratio determination model 5A for calculating the existing ratio for each grade of the scrap is generated from the image including the scrap aggregate 11.

[0051] [Collection Recognition Model] Figure 4 shows examples of images used in machine learning as input data and training data when generating the collection recognition model 5B. Figure 4(a) shows an image containing the scrap collection 11, which is the input data. The image containing the scrap collection 11 does not need to contain the entire scrap collection 11, but it is preferable that the entire scrap collection 11 is contained within the image. Figure 4(b) shows the recognition of the scrap collection 11 from the image containing the scrap collection 11 using an object detection model based on the features of the target object. Examples of object detection models include YOLO (You Only Look Once) and SSD (Single Shot Multibox Detector). Figure 4(c) shows that, after identifying the scrap aggregate 11 using the object detection model, further detailed information about the scrap aggregate 11, such as its shape and contour, is identified using the segmentation model. Figure 4(d) shows that the scrap aggregate 11 has been identified and serves as training data indicating the precise location information of the scrap aggregate 11.

[0052] Therefore, machine learning is performed using an image containing the scrap aggregate 11 as input data and the precise location information of the scrap aggregate 11 as training data. This generates an aggregate recognition model 5B for obtaining the precise location information of the scrap aggregate 11 from an image containing the scrap aggregate 11. Alternatively, the object detection model or segmentation model used to generate the aggregate recognition model 5B may also be used as the aggregate recognition model 5B.

[0053] The proportion determination program in this embodiment is executed by the processor to sequentially or repeatedly perform the following processes as needed. First, it performs a process to acquire an image containing the object collection. Next, it performs a process to acquire location information of the object collection or transport device. Furthermore, it performs a process to set one or more shooting locations based on the acquired location information and take pictures. In addition, it performs a process to calculate the presence ratio of each type of object in the object collection for each image taken using the proportion determination model 5A. Finally, it performs a process to calculate the presence ratio of each type of object in the object collection from all the images from which the calculation process has been performed.

[0054] The ratio determination program in this embodiment can be executed by any information processing device equipped with a CPU, RAM, ROM, storage, input / output interface, and network interface.

[0055] The recording medium storing the ratio determination program in this embodiment contains a sequence of instructions for a computer to execute the ratio determination program in this invention. When the processor executes this sequence of instructions, a series of processes are performed to calculate the proportion of each type of target object based on the captured image.

[0056] [Method] The imaging device 2 and the control unit of the calculation device 3 function as the imaging unit 2A, position information acquisition unit 2B, shooting location setting unit 2C, camera control unit 2D, image acquisition unit 3A, ratio calculation unit 3B, overall calculation unit 3C, and calculation unit 3D by executing the processes shown in Figure 5 according to the program.

[0057] In step S1, the imaging unit 2A processes an image containing the scrap assemblies 11 in the scrap yard using the camera 21. Figure 6(a) shows an example of the scrap assemblies 11 being photographed by the camera 21. Specifically, since the camera 21 continuously photographs the scrap yard, an image containing the scrap assemblies 11 can be captured as soon as the scrap assemblies 11 are brought in.

[0058] In step S2, the location information acquisition unit 2B uses the trained aggregate recognition model 5B to acquire location information of the scrap aggregate 11 from an image containing the scrap aggregate 11. Figure 6(b) shows an example of acquiring location information of the scrap aggregate 11, and the image has been enlarged so that the scrap aggregate 11 fits into a single image.

[0059] In step S3, the shooting location setting unit 2C sets one or more shooting locations from the acquired location information of the scrap assembly 11. Figure 6(c) shows an example of setting shooting locations for the scrap assembly 11. Specifically, the circumscribing rectangle of the scrap assembly 11 is divided into six grid sections, and images 11a, 11b, 11c, 11d, 11e, and 11f are set as shooting locations.

[0060] In step S4, the camera control unit 2D controls the shooting direction and magnification of the camera 21 for each of the set shooting locations and takes a picture. In the order of the arrows shown in Figure 6(c), the entire scrap assembly 11 is photographed by optical zooming on each of the six images 11a to 11f obtained by dividing the scrap assembly 11 into a grid.

[0061] Figure 7 shows a comparative example of an enlarged image of a portion of the scrap assembly 11. Figure 7(a) is image 11g obtained by enlarging and then dividing the overall image of the scrap assembly 11. Figure 7(b) is image 11h taken by enlarging a set shooting area using optical zoom or the like. As shown in Figure 7(a), it is difficult to obtain high-quality images when the image is divided after enlarging the overall image. On the other hand, as shown in Figure 7(b), high-quality images can be obtained by using the method of taking an image after enlarging a set shooting area using optical zoom or the like. The higher the quality of the acquired image, the higher the accuracy of the existence ratio calculated by the ratio calculation unit 3B. Specifically, it is desirable that the pixel resolution be 1.2 mm / pixel or less.

[0062] In step S5, the image acquisition unit 3A acquires images 11a to 11f that include the scrap aggregate 11 that has been captured by the imaging device 2.

[0063] In step S6, the ratio calculation unit 3B uses the trained ratio determination model 5A to calculate the proportion of each grade of scrap for each of the images 11a to 11f acquired for each shooting location. Figure 6(d) shows an example of obtaining determination results for each of the images 11a to 11f.

[0064] In step S7, the ratio calculation unit 3B checks whether it has calculated the proportion of each grade of scrap for all images acquired for each shooting location in the scrap assembly 11. If the ratio calculation unit 3B has not been able to obtain a calculation result for all images acquired for each shooting location in the scrap assembly 11, the process returns to step S4. Then, the camera control unit 2D takes another photograph of the shooting location set by the shooting location setting unit 2C. Note that the shooting location may be set again, and there is no limit to the number of attempts to take another photograph. On the other hand, if the ratio calculation unit 3B has been able to obtain a calculation result for all images acquired for each shooting location in the scrap assembly 11, the process proceeds to step S8.

[0065] In step S8, the overall calculation unit 3C calculates the proportion of each grade of scrap in the scrap assembly 11 based on the proportion of each grade of scrap calculated for each acquired image. In other words, it calculates the proportion of each grade of scrap in the scrap assembly 11 based on the proportion of each grade of scrap calculated for each of the images 11a to 11f acquired for each shooting location. Specifically, it further calculates the average value of the proportion of each grade of scrap in each of the calculated images 11a to 11f, and uses that average value as the proportion of each grade of scrap in the scrap assembly 11.

[0066] Subsequently, the calculation unit 3D calculates the price of the scrap assembly 11 based on the proportion of each grade of scrap present in the scrap assembly 11 and the unit price of each grade of scrap.

[0067] Generally, scrap assemblies 11 containing a mixture of scrap of various grades are transported to steel mills that use electric furnaces or blast furnaces. The present invention makes it possible to calculate the proportion of each scrap grade even in scrap assemblies 11 containing a mixture of various grades. Furthermore, the present invention makes it possible to quickly calculate the price of scrap assemblies 11 containing a mixture of various grades based on that proportion.

[0068] Below are some examples of measures to avoid overlapping or overlooking shooting locations. For example, while the scrap pile 11 is stationary, it is possible to pre-determine the shooting locations based on the location information of the scrap pile 11 and the camera's shooting range before shooting. Alternatively, by memorizing the areas of the scrap pile 11 that have been photographed, it is possible to limit shooting to areas of the scrap pile 11 that have not yet been photographed.

[0069] [Second Embodiment] [Configuration] The configuration of the ratio determination system 1 in the second embodiment will be described below. The second embodiment differs from the first embodiment in that the scrap aggregate 11 is moved by external factors, such as being transported by a belt conveyor or lifting magnet. However, the same reference numerals are used for the same devices, structures, etc., to avoid redundant explanations.

[0070] If the movement of the scrap pile 11 is random, it is not necessary to photograph the entire scrap pile 11. The locations to be photographed are not particularly limited; for example, one location, two, three, or four or more arbitrary locations may be photographed.

[0071] Next, if the scrap pile 11 moves periodically, such as when transported on a conveyor belt, the photography may be repeated in accordance with that period. For example, when the scrap pile 11 is being transported on a conveyor belt in a constant direction at a constant speed, when a portion of the scrap pile 11 that has been photographed moves out of frame, another portion of the scrap pile 11 that has not yet been photographed moves in and enters frame. The photography of a portion of the scrap pile 11 is repeated until the transport of the scrap pile 11 is complete.

[0072] Finally, if the scrap aggregate 11 is transported and moved using a transport device, such as by a lifting magnet, the transport device may be used as the reference point for photography to determine the location to be photographed. As described above, the photography of a portion of the scrap aggregate 11 is repeated until the transport of the scrap aggregate 11 is complete.

[0073] [Method] The imaging unit 2A photographs a transport device, such as a lifting magnet, that transports a portion of the scrap assembly 11. For example, to photograph scrap being transported, a lifting magnet that transports a portion of the scrap assembly 11 is used as the shooting reference. To use the lifting magnet as the shooting reference, the position information acquisition unit 2B acquires the position information of the lifting magnet. Then, the shooting location setting unit 2C sets a portion of the scrap assembly 11 located at a predetermined distance from the lifting magnet as the shooting location, based on the position information of the lifting magnet. Finally, the camera control unit 2D controls the camera's shooting direction and magnification based on the set shooting location. The above series of operations is repeated until the entire scrap assembly 11 has been transported.

[0074] Furthermore, in order to use lifting magnets as a reference point for photography, it is necessary to acquire the positional information of the lifting magnets. There are no particular limitations on the method of acquiring the positional information of lifting magnets, but for example, a transport device detection model that detects lifting magnets from images that include lifting magnets transporting scrap metal may be used. Using a transport device detection model, the positional information of lifting magnets can be acquired based on the equipment configuration of the scrap yard, etc. Alternatively, signals indicating the positional information of lifting magnets emitted by the lifting magnet control device may be acquired.

[0075] [Storage Location Selection Method] A method for selecting the storage location of an object assembly using the proportion determination system 1 will be described. The method for selecting the storage location of an object assembly using the proportion determination system 1 involves determining the types of objects in the object assembly based on the proportion of each type of object present in the object assembly calculated using the proportion determination system 1. Then, the object assembly is transported to the storage location determined for each type of object according to that determination. For example, the grade of the scrap in the scrap assembly 11 is determined based on the proportion of each grade of scrap present in the scrap assembly 11 calculated using the proportion determination system 1 in the second embodiment. Then, the scrap assembly 11 is transported to the storage location provided for each grade of scrap according to that grade.

[0076] Specifically, a storage area may be set up for each grade of scrap, and the scrap grade with the highest proportion among the calculated proportions of each scrap grade in the scrap assembly 11 may be designated as the representative grade of the scrap assembly 11, and the storage area may be determined based on that representative grade. In this case, by setting location information in advance for each storage area, the scrap assembly 11 can be automatically transported to the storage area corresponding to the representative grade using a transport device that transports the scrap assembly 11 to each location information.

[0077] Furthermore, the storage locations are not particularly limited, but for example, they may be set according to the grade of the scrap, or according to the various attributes of the scrap. The various attributes of the scrap include the price for each type of scrap, the composition of each type of scrap, and whether the scrap assembly 11 is acceptable or not.

[0078] Furthermore, while the determination of the storage location is not particularly limited, for example, the storage location may be determined based on the grade of scrap with the highest proportion among the proportions of each grade of scrap present in the calculated scrap assembly 11. In addition, the product of the proportion of each grade of scrap present in the calculated scrap assembly 11 and the price corresponding to that grade may be calculated, and the grade with the highest value of this product may be designated as the representative grade of the scrap assembly 11, and the storage location may be determined based on that representative grade. Moreover, different storage locations may be determined depending on whether the scrap assembly 11 is approved or rejected.

[0079] Because scrap of various grades is mixed together, undesirable materials such as low-grade scrap and foreign matter may be present in the furnace. This invention allows for periodic inspections to ensure that the proportion of low-grade scrap and foreign matter remains below a certain standard.

[0080] [Modifications] The present invention has been described above with reference to specific embodiments, but these descriptions are not intended to limit the invention. By referring to the description of the present invention, embodiments of the present invention, including various modifications, will be obvious to those skilled in the art along with the disclosed embodiments. Accordingly, the embodiments of the invention described in the claims should be understood to include embodiments that include these modifications described herein, either individually or in combination.

[0081] In the above embodiment, the proportion of existence was defined as a weight proportion, but the present invention is not limited to this example. For example, the proportion of existence may be any other proportion, such as an area proportion or a volume proportion.

[0082] Furthermore, in the above embodiment, when generating the ratio determination model 5A, the image containing the scrap collection 11 as input data is divided, but the present invention is not limited to this example. For example, division is not required.

[0083] Furthermore, in the above embodiment, when generating the ratio determination model 5A, the image containing the scrap collection 11 as input data was divided into three parts, but the present invention is not limited to this example. For example, the number of divisions may be two, four or more.

[0084] Furthermore, in the above embodiment, when generating the ratio determination model 5A, the image containing the scrap collection 11 as input data was divided vertically, but the present invention is not limited to this example. For example, the division method may be horizontal, or a grid, etc.

[0085] Furthermore, in the above embodiment, the method for creating training data in the aggregate recognition model 5B is not particularly limited. For example, both an object detection model and a segmentation model may be used, or training data may be created using only one of the object detection model and the segmentation model. Moreover, even without using the above models, training data may be created by manual methods, such as drawing a rectangle around the scrap aggregate 11 or coloring along the shape and contour of the scrap aggregate 11.

[0086] Furthermore, in the above embodiment, the type of camera used in the imaging unit 2A is not particularly limited. For example, the camera may be a video camera that generates moving images, or a still camera that generates still images.

[0087] Furthermore, the number of cameras in the above embodiment is not particularly limited. For example, there may be one camera, two, three, or four or more cameras. Therefore, multiple images may be used by capturing images with multiple fixed cameras.

[0088] Furthermore, the camera in the above embodiment is not particularly limited. For example, a PTZ camera capable of panning is preferred because it can flexibly change the shooting range to acquire images.

[0089] Furthermore, in the above embodiment, the image captured by the imaging unit 2A was used to calculate the proportion of each grade of scrap in the scrap aggregate 11, but the present invention is not limited to this example. For example, the image captured by the imaging unit 2A may be used for machine learning of the proportion determination model 5A and the aggregate recognition model 5B.

[0090] Furthermore, in the above embodiment, the shooting location setting unit 2C divides the entire outline of the scrap collection 11 into six grid-like sections, but the present invention is not limited to this example. For example, the shooting location setting unit 2C may divide the image evenly into grid sections, or it may cut out sections randomly. Also, it is not necessary to cover the entire scrap collection 11 as setting locations; only a representative part may be set as the setting locations.

[0091] Furthermore, in the above embodiment, the camera control unit 2D captures images of the scrap collection 11 divided into six grid sections, but the present invention is not limited to this example. For example, the camera control unit 2D may sequentially capture the set shooting locations while changing the shooting location, or it may simultaneously capture different shooting locations with multiple cameras.

[0092] Furthermore, in the above embodiment, the image acquisition unit 3A acquires an image that includes the scrap collection 11 that has been photographed by the photographing device 2, but the present invention is not limited to this example. For example, the image acquisition unit 3A may acquire an image each time the photographing device 2 takes a picture, or it may acquire an image when a certain number of photographed images have been collected, or it may acquire all images after all photographing is completed.

[0093] Furthermore, in the above embodiment, the scrap grades for which the proportion calculation unit 3B calculates the proportion of existence were set to three types: grade A, grade B, and grade C. However, the present invention is not limited to such examples. For example, the scrap grades for which the proportion calculation unit 3B calculates the proportion of existence may be one type, two types, or four or more types. Moreover, the scrap grade may be a single grade, such as grade A being 100%, grade B 0%, and grade C 0%. Alternatively, the scrap grade may be such that the proportion of existence for a certain grade is 0%, such as grade A being 80%, grade B 20%, and grade C 0%.

[0094] Furthermore, in the above embodiment, the calculation method by the overall calculation unit 3C was the arithmetic mean, but the present invention is not limited to such examples. For example, the calculation method may be a weighted average. A weighted average is a method of averaging data for each grade by assigning weights to them, and is used when different importance exists for each grade. For example, referring to Figure 2, in image 11A, the proportion of scrap in the image is small, and conversely, the proportion of blank space is large. On the other hand, in image 11B, the proportion of scrap in the image is large, and conversely, the proportion of blank space is small. In such a case, in order to accurately calculate the proportion of each grade of scrap in the scrap aggregate 11, it is conceivable to make the specific gravity of image 11B heavier than that of image 11A, and then average the values ​​for each image calculated based on the specific gravity.

[0095] Furthermore, in the above embodiment, the calculation unit 3D calculates the price of the scrap aggregate 11 based on the unit price for each grade of scrap, but the present invention is not limited to such examples. For example, the calculation unit 3D may calculate the component composition of the scrap aggregate 11 based on the component composition for each grade and brand of scrap. Here, the brand of scrap refers to the type of scrap itself, and examples include reinforcing bars and steel sheet piles. Furthermore, the calculation unit 3D may determine whether the scrap aggregate 11 is acceptable or unacceptable based on judgment criteria for sorting the scrap aggregate 11 into acceptable and unacceptable categories. Note that the calculation unit 3D stores the above unit price, component composition, judgment criteria, etc., but it may also store other information or delete it from storage.

[0096] Furthermore, in the above embodiment, the scrap to be photographed is not particularly limited. For example, the scrap to be photographed may be a portion of the scrap being transported by the lifting magnet. Alternatively, the scrap to be photographed may be a portion of the remaining scrap that newly appears on the surface after a portion of the scrap has been transported.

[0097] [Other] The present invention may also have the following configurations.

[0098] (1) A calculation device for calculating the proportion of each type of object in a collection of objects captured by a shooting device, wherein the calculation device has a proportion calculation unit that calculates the proportion of each object using a pre-trained model generated by machine learning, with the image containing the collection of objects as input data and the proportion of each type of object in the collection of objects as training data.

[0099] (2) A device for generating a proportion determination model, characterized in that the device generates a trained model by machine learning using an image containing a collection of objects as input data and the proportion of each type of object in the collection of objects as training data.

[0100] (3) A method for generating a proportion determination model, characterized in that the generation method generates a trained model by machine learning using an image containing the object collection as input data and the proportion of each type of object in the object collection as training data.

[0101] (4) A program that causes a computer to execute a method for generating a proportion determination model, characterized in that the program generates a trained model by machine learning using an image containing the object collection as input data and the proportion of each type of object in the object collection as training data.

[0102] 1. Proportion determination system 2. Imaging device 3. Calculation unit 4. Learning device 5. Storage device 2A. Imaging unit 2B. Location information acquisition unit 2C. Imaging location setting unit 2D. Camera control unit 3A. Image acquisition unit 3B. Proportion calculation unit 3C. Overall calculation unit 3D. Calculation unit 4A. Image acquisition unit 4B. Image processing unit 4C. Machine learning unit 5A. Proportion determination model 5B. Aggregate recognition model 11. Scrap aggregate 11A-C. Divided images of the scrap aggregate 11a-h. Divided images of the scrap aggregate 21. Camera

Claims

1. A ratio determination system comprising: an imaging device that captures an image containing a collection of multiple types of objects; and a calculation device that calculates the proportion of each type of object present in the collection of objects contained in the image, wherein the calculation device has a ratio calculation unit that calculates the proportion of each object present using a pre-trained model generated by machine learning, with the image containing the collection of objects as input data and the proportion of each type of object present in the collection of objects as training data.

2. The proportion determination system according to claim 1, wherein the imaging device has a position information acquisition unit that acquires position information of the object collection, and the position information acquisition unit acquires position information of the object collection from the image containing the object collection using a pre-trained model generated by machine learning with the image containing the object collection as input data and the position information of the object collection as training data.

3. The proportion determination system according to claim 1, wherein the imaging device has a position information acquisition unit that acquires position information of a transport device that transports the object assembly, and the position information acquisition unit acquires the position information of the transport device from the image containing the transport device using a pre-trained model generated by machine learning with the image containing the transport device as input data and the position information of the transport device as training data.

4. The proportion determination system according to claim 2, characterized in that the imaging device has an imaging location setting unit that sets one or more imaging locations based on the acquired position information of the object collection.

5. The ratio determination system according to claim 3, characterized in that the imaging device has an imaging location setting unit that sets one or more imaging locations based on the acquired position information of the transport device.

6. The ratio determination system according to claim 4 or 5, characterized in that the imaging device has a camera control unit that controls the camera's imaging direction and magnification based on the imaging location set by the imaging location setting unit.

7. The ratio determination system according to claim 6, characterized in that the ratio calculation unit calculates the proportion of each type of target object for each shooting location.

8. The ratio determination system according to claim 7, characterized in that the calculation device has an overall calculation unit that calculates the proportion of each type of target object in the collection of target objects based on the proportion of each type of target object calculated for each shooting location.

9. The proportion determination system according to claim 1, characterized in that the calculation device has a calculation unit that stores the unit price for each type of object, and the calculation unit calculates the price of the object collection based on the proportion of existence and the unit price.

10. The proportion determination system according to claim 1, wherein the calculation device has a calculation unit that stores the component composition for each type of target object, and the calculation unit calculates the component composition of the object aggregate based on the proportion of existence and the component composition.

11. The proportion determination system according to claim 1, wherein the calculation device has a calculation unit that stores judgment criteria for classifying the object collection into pass and fail, and the calculation unit determines whether the object collection is pass or fail based on the proportion of existence and the judgment criteria.

12. The proportion determination system according to claim 1, characterized in that the object in question is scrap.

13. A method for determining proportions, comprising the steps of: taking an image containing a collection of objects of multiple types; and calculating the proportion of each type of object in the collection of objects contained in the image, wherein the method is characterized in that it calculates the proportion of each type of object using a pre-trained model generated by machine learning, with the image containing the collection of objects as input data and the proportion of each type of object in the collection of objects as training data.

14. A proportion determination program that causes a computer to take an image containing a collection of objects of multiple types, and to calculate the proportion of each type of object in the collection of objects contained in the image, wherein the proportion of each type of object is calculated using a pre-trained model generated by machine learning, with the image containing the collection of objects as input data and the proportion of each type of object in the collection of objects as training data.

15. A computer-readable recording medium having stored a program that causes a computer to take an image containing a collection of objects of multiple types, and to calculate the proportion of each type of object in the collection of objects contained in the image, wherein the proportion of each type of object is calculated using a pre-trained model generated by machine learning, with the image containing the collection of objects as input data and the proportion of each type of object in the collection of objects as training data.

16. A method for selecting a storage location, characterized by determining the type of object in the object collection based on the proportion of each type of object in the object collection calculated using the proportion determination system described in claim 1, and transporting the object collection to a designated storage location for each type of object according to the determination.

Images