Aggregate determination device and aggregate determination method

Abstract

To provide an aggregate determination device and an aggregate determination method that can accurately determine whether an aggregate introduced to a conveyor matches aggregate types input to a control panel regardless of if a surface of the aggregate is in a dry state or a wet state.SOLUTION: An aggregate determination device 50 comprises a control panel 51, a sprinkler nozzle 58, a camera 53, and an information processing unit 56. The sprinkler nozzle 58 sprinkles water on an aggregate 9. The camera 53 photographs the aggregate 9 after the sprinkling. The information processing unit 56 estimates an aggregate type based on an image of the aggregate 9 obtained through the photographing performed by the camera 53. The information processing unit 56 determines whether a result of the estimation matches information on aggregate types input to the control panel 51. The sprinkling allows a surface of the aggregate to enter a uniformly wet state. Consequently, the information processing unit 56 can obtain an accurate determination result.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an aggregate determination device and an aggregate determination method for determining whether aggregate fed onto a conveyor is suitable. [Background technology]

[0002] Conventionally, concrete plants are provided with multiple silos for storing aggregates, which are raw materials for concrete. Aggregates transported by truck from a quarry are transported by conveyors within the concrete plant and stored in designated silos for each type of aggregate. A conventional concrete plant is described, for example, in Patent Document 1. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 08-091580 Summary of the Invention [Problem to be solved by the invention]

[0004] Aggregates are classified into several types according to characteristics such as particle size, color, and place of origin. In concrete plants, a specific silo is designated for storing each type of aggregate. When aggregate is dumped from a truck onto a conveyor, the truck driver or a designated worker inputs the aggregate type into a control panel. This sets the destination of the aggregate by the conveyor to the silo corresponding to that aggregate type.

[0005] If the wrong aggregate is fed onto the conveyor, or if the wrong type of aggregate is entered into the control panel, the fed aggregate will be transported to the wrong silo. This will result in the silo being mixed with aggregate of a different type. In this case, it will be impossible to properly control the quality of the ready-mix concrete produced, and it will be necessary to remove all of the aggregate from the silo where the mixing occurred. This will result in a great deal of work and damage.

[0006] In order to solve the above problems, for example, it is conceivable to photograph the aggregate input to the conveyor and, based on the obtained image, determine by a computer whether or not the aggregate matches the aggregate type input to the operation panel. However, the color of the surface of the aggregate input to the conveyor changes depending on whether it is in a dry state or a wet state. For this reason, there is a problem that it is difficult to accurately determine the aggregate type based on the image of the aggregate.

[0007] The present invention has been made in view of such circumstances, and an object thereof is to provide an aggregate determination device and an aggregate determination method capable of accurately determining whether or not an aggregate input to a conveyor matches the aggregate type input to an operation panel, regardless of whether the surface of the aggregate input to the conveyor is in a dry state or a wet state.

Means for Solving the Problems

[0008] To solve the above problems, a first invention of the present application is an aggregate determination device for determining the suitability of an aggregate input to a conveyor, comprising: a watering nozzle for watering the aggregate; a camera for photographing the aggregate conveyed by the conveyor on the downstream side of the conveyance path from the watering nozzle; an operation panel for inputting information on the aggregate type; and an information processing unit communicably connected to the camera and the operation panel, wherein the information processing unit inputs an image of the aggregate obtained by photographing with the camera into a learning model generated by machine learning, and outputs an estimation result of the aggregate type from the learning model, and a determination unit for determining whether or not the aggregate type input to the operation panel matches the estimation result.

[0009] A second invention of the present application is the aggregate determination device according to the first invention, further comprising: a moisture sensor for measuring the moisture content on the surface of the aggregate conveyed by the conveyor; and a control unit for switching the presence or absence of watering or the amount of watering from the watering nozzle according to the measured value of the moisture sensor.

[0010] The third invention of the present application is the aggregate determination device of the second invention, wherein the moisture sensor is located on the downstream side of the conveying path from the water spray nozzle.

[0011] The fourth invention of the present application is the aggregate determination device according to any one of the first to third inventions, wherein the camera captures a moving image of the aggregate conveyed by the conveyor, the moving image includes a plurality of frame images having blurring in the conveying direction of the aggregate, and the information processing unit inputs the frame images included in the moving image captured by the camera into the learning model.

[0012] The fifth invention of the present application is the aggregate determination device of the fourth invention, wherein the information processing unit further has a learning unit that generates the learning model by performing machine learning using, as teacher data, a plurality of frame images included in a moving image of a sample aggregate in a wet state prepared for learning and the known aggregate type of the sample aggregate.

[0013] The sixth invention of the present application is the aggregate determination device of the fifth invention, wherein the operation panel can input correct / error information indicating whether the determination result by the determination unit is correct, and the learning unit updates the learning model by performing additional machine learning using, as teacher data, the image of the aggregate obtained from the camera, the estimation result, and the correct / error information.

[0014] The seventh invention of the present application is the aggregate determination device according to any one of the first to sixth inventions, wherein the estimation unit inputs an odd number of images obtained from the camera into the learning model, outputs an odd number of estimation results corresponding to each of the odd number of images from the learning model, and the determination unit compares the aggregate type input to the operation panel with the odd number of estimation results and makes a determination based on a majority vote of the comparison results.

[0015] The eighth invention of the present application is an aggregate determination method for determining the suitability of aggregates input to a conveyor, comprising: a) a step of inputting information on the aggregate type of the aggregates input to the conveyor into an operation panel; b) a step of spraying water on the aggregates; c) a step of photographing the aggregates conveyed by the conveyor after the step b); d) a step of inputting an image of the aggregates obtained by the photographing in the step c into a learning model generated by machine learning and outputting an estimated result of the aggregate type from the learning model; and e) a step of determining whether or not the aggregate type input in the step a matches the estimated result.

Effect of the Invention

[0016] According to the first to eighth inventions of the present application, by spraying water on the aggregates input to the conveyor, the surface of the aggregates is uniformly wetted. Then, based on the image of the wetted aggregates, the aggregate type is estimated. Thereby, it is possible to accurately determine whether or not the aggregates input to the conveyor match the aggregate type input to the operation panel.

[0017] In particular, according to the second invention of the present application, the moisture content on the surface of the aggregates can be made substantially constant. Thereby, the estimation accuracy of the aggregate type can be further improved.

[0018] In particular, according to the third invention of the present application, the moisture sensor measures the moisture content on the surface of the aggregates after spraying. Thereby, the moisture content of the aggregates at the time of photographing by the camera can be grasped more accurately. Therefore, it is easy to manage by associating the image with the moisture content.

[0019] In particular, according to the fourth invention of the present application, a moving image of the aggregates is photographed, and a frame image having blur in the conveying direction is input to the learning model. The camera for photographing the aggregates does not need to be a high-speed camera with a high frame rate capable of acquiring a frame image without blur in the conveying direction or a high-performance camera with a large number of pixels, and an inexpensive camera that is easily available is sufficient. Thereby, the cost of the camera can be reduced. Also, by using an inexpensive camera that is easily available, the replacement cost when the camera fails can be suppressed. Therefore, an aggregate determination device excellent in maintainability can be realized.

[0020] In particular, according to the fifth invention of the present application, a large number of frame images included in a video are used as learning images. Therefore, a large number of aggregate images required for generating a learning model can be acquired in a relatively short time. In addition, a learning model having sufficient estimation accuracy can be created from frame images having blur. Further, compared with the case of preparing a large number of detailed images, the storage capacity of the aggregate determination device can be reduced. That is, a learning model with high estimation accuracy can be generated with a low-cost device configuration.

[0021] In particular, according to the sixth invention of the present application, the learning model can be updated by performing additional machine learning. Thereby, the estimation accuracy of the aggregate type by the learning model can be maintained at a certain level. In addition, by accumulating aggregate images, estimation results, and correct / error information, additional machine learning can be performed quickly.

[0022] In particular, according to the seventh invention of the present application, a more accurate estimation result can be obtained based on an odd number of images.

Brief Description of the Drawings

[0023] [Figure 1] It is a diagram showing the configuration of an aggregate conveying device according to the first embodiment. [Diagram 2] It is a block diagram conceptually showing the functions realized in the information processing unit of the first embodiment. [Figure 3] It is a flowchart showing the flow of learning processing in the first embodiment. [Figure 4] It is a diagram showing an example of a learning image cut out from a video. [Figure 5] It is a flowchart showing the flow of determination processing in the first embodiment. [Figure 6] It is a diagram showing the configuration of an aggregate conveying device according to the second embodiment. [Figure 7] It is a flowchart showing the flow of determination processing in the second embodiment. [Figure 8]It is a flowchart showing the flow of the determination process in the third embodiment. [Figure 9] It is a flowchart showing the flow of the process when performing additional machine learning in the fourth embodiment.

Mode for Carrying Out the Invention

[0024] Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

[0025] <1. First Embodiment> <1-1. Configuration of Aggregate Conveyor> FIG. 1 is a diagram showing the configuration of an aggregate conveyor 1 including an aggregate determination device 50. This aggregate conveyor 1 is a device that receives aggregate 9 as a raw material in a concrete plant and conveys the aggregate 9 to a plurality of silos 40. As shown in FIG. 1, the aggregate conveyor 1 includes a receiving hopper 10, a receiving conveyor 20, a shuttle conveyor 30, four silos 40, and an aggregate determination device 50.

[0026] The receiving hopper 10 is disposed above the conveying start position of the receiving conveyor 20. The receiving hopper 10 is a funnel-shaped hopper whose inner diameter gradually decreases downward. The aggregate 9 is transported from a quarry by a truck 90 and is introduced into the upper opening of the receiving hopper 10. The introduced aggregate 9 is temporarily stored in the receiving hopper 10 and is supplied from the lower opening of the receiving hopper 10 to the upper surface of the receiving conveyor 20.

[0027] The receiving conveyor 20 is a mechanism that conveys the aggregate 9 introduced into the receiving hopper 10 obliquely upward. The receiving conveyor 20 has a starting pulley 21, a terminal pulley 22, and an annular belt 23 spanned between these pulleys 21 and 22. The starting pulley 21 is located below the receiving hopper 10. The terminal pulley 22 is located above a shuttle conveyor 30 described later. The terminal pulley 22 is disposed at a position obliquely upward and away from the starting pulley 21.

[0028] At least one of the start pulley 21 and the end pulley 22 is rotated by the power of a motor (not shown). As a result, a belt 23 rotates between the start pulley 21 and the end pulley 22 in the direction of the arrow in FIG. 1. The moving speed of the belt 23 is adjusted to any speed, for example, 50 m / min or higher. The aggregates 9 fed into the receiving hopper 10 are supplied to the upper surface of the belt 23 near the start pulley 21. Then, as the belt 23 rotates, the aggregates 9 placed on the belt 23 move obliquely upward from a position below the receiving hopper 10 toward the end pulley 22.

[0029] The belt 23 of the receiving conveyor 20 moves continuously at a constant speed. Therefore, the aggregates 9 drawn from the bottom of the receiving hopper 10 are transported while being placed at a substantially constant height on the upper surface of the belt 23. When the aggregates 9 are transported to the position of the terminal pulley 22, the belt 23 reverses and the aggregates 9 fall onto the shuttle conveyor 30.

[0030] The shuttle conveyor 30 is a mechanism for distributing the aggregates 9 transported from the receiving conveyor 20 to the four silos 40. The shuttle conveyor 30 is located below the terminal pulley 22 of the receiving conveyor 20. The shuttle conveyor 30 has a first pulley 31, a second pulley 32, and a circular belt 33 stretched between the pulleys 31 and 32.

[0031] At least one of the first pulley 31 and the second pulley 32 is rotated by the power of a motor (not shown). This causes the belt 33 to rotate between the first pulley 31 and the second pulley 32. The aggregates 9 that have fallen from the receiving conveyor 20 and placed on the belt 33 of the shuttle conveyor 30 are transported to one of the four silos 40 by the rotation of the belt 33.

[0032] The shuttle conveyor 30 is movable along a horizontal plane by a drive mechanism (not shown), which allows the destination of the aggregate 9 by the shuttle conveyor 30 to be switched to a specified one of the four silos 40.

[0033] The four silos 40 are storage tanks for the aggregate 9. The aggregate 9 is classified into a plurality of aggregate types according to characteristics such as particle size, color, and place of origin. In this aggregate conveying device 1, a silo 40 to be stored is determined for each aggregate type. In the example of FIG. 1, the aggregates 9 of four aggregate types A to D are separately stored in the four silos 40. Note that the number of silos 40 provided in the aggregate conveying device 1 is not limited to four.

[0034] <Regarding the aggregate determination device> The aggregate determination device 50 is a device for determining the suitability of the aggregate 9 input to the receiving conveyor 20. The aggregate determination device 50 photographs the aggregate 9 conveyed by the receiving conveyor 20 and estimates the aggregate type based on the obtained image. Then, the aggregate determination device 50 determines whether the estimated result of the aggregate type matches the specified aggregate type. As shown in FIG. 1, the aggregate determination device 50 includes an operation panel 51, a display unit 52, a watering nozzle 58, a camera 53, a light source 54, a hood 55, and an information processing unit 56.

[0035] The operation panel 51 is disposed in the vicinity of the receiving hopper 10. The operation panel 51 is electrically connected to the information processing unit 56. The operation panel 51 has a plurality of keys 511, 512. An operator (the driver of the truck 90 or a worker of the concrete plant) presses a key 511 indicating the aggregate type of the aggregate 9 input to the receiving hopper 10. Thereby, the information of the aggregate type is input from the operation panel 51 to the information processing unit 56. Further, when the information of the aggregate type is input, the shuttle conveyor 30 changes its position so that the silo 40 of the said aggregate type becomes the conveyance destination.

[0036] Also, after the aggregate 9 is input to the receiving hopper 10, the operator presses a key 512 for instructing the start of conveyance. Then, the operations of the receiving conveyor 20 and the shuttle conveyor 30 are started. Thereby, the input aggregate 9 is conveyed to the specified silo 40 by the receiving conveyor 20 and the shuttle conveyor 30.

[0037] Note that the operation panel 51 may be configured by a general keyboard, mouse, or touch panel.

[0038] The display unit 52 is arranged near the receiving hopper 10 together with the operation panel 51. The display unit 52 is electrically connected to the information processing unit 56. For example, a liquid crystal display is used for the display unit 52. The display unit 52 displays various information regarding the aggregate determination device 50. For example, the display unit 52 displays the determination result output from the determination unit 63 (to be described later) of the information processing unit 56.

[0039] The watering nozzle 58 is a nozzle that waters the aggregate 9 conveyed by the receiving conveyor 20. The watering nozzle 58 is located on the downstream side of the conveying path from the receiving hopper 10 and on the upstream side of the conveying path from the camera 53. The watering nozzle 58 is connected to a water supply source such as a water supply. Also, a valve 581 is provided in the flow path between the watering nozzle 58 and the water supply source. When the valve 581 is opened, the water supplied from the water supply source is discharged from the watering nozzle 58. Thereby, the surface of the aggregate 9 becomes wet with water.

[0040] The camera 53 is an imaging device that photographs the aggregate 9 conveyed by the receiving conveyor 20. The camera 53 is arranged so as to face the upper surface of the belt 23 of the receiving conveyor 20 on the downstream side of the conveying path from the watering nozzle 58. The camera 53 of the present embodiment photographs a moving image of the aggregate 9 conveyed by the receiving conveyor 20. After the operation of the receiving conveyor 20 is started and a predetermined time has elapsed and the leading aggregate 9 passes below the camera 53, the camera 53 starts photographing the moving image. Then, the camera 53 inputs the obtained moving image to the information processing unit 56.

[0041] Note that a sensor for detecting the aggregate 9 may be provided in the middle of the receiving conveyor 20, and the photographing of the moving image by the camera 53 may be started based on the detection signal of the sensor.

[0042] The light source 54 is a part for irradiating light toward the imaging range of the camera 53. For the light source 54, for example, an LED lamp is used. However, the light source 54 may be a fluorescent lamp or an incandescent lamp. The light source 54 is always lit when the camera 53 performs imaging.

[0043] The hood 55 is a box-shaped member that covers a part of the upper surface of the receiving conveyor 20. The hood 55 is formed of a light-shielding material. The hood 55 is disposed in the middle of the conveying path of the aggregate 9 by the receiving conveyor 20. The hood 55 has an inlet opening 551 and an outlet opening 552. The aggregate 9 conveyed by the receiving conveyor 20 is carried into the hood 55 through the inlet opening 551 and carried out of the hood 55 through the outlet opening 552.

[0044] The camera 53 and the light source 54 are disposed inside the hood 55. External ambient light is shielded by the hood 55. Thereby, the influence of ambient light during imaging by the camera 53 is suppressed. Also, inside the hood 55, by irradiating light from the light source 54, the aggregate 9 can be illuminated to a certain brightness. Therefore, even when the solar radiation amount changes due to the season or weather, the aggregate 9 can be imaged at a certain brightness. Also, it is possible to suppress the change in the shadow of the aggregate 9 due to the position of the sun.

[0045] The information processing unit 56 is a device that performs the learning process and the determination process described later. The information processing unit 56 is constituted by a computer having a processor 561 such as a CPU or a GPU, a memory 562 such as a RAM, and a storage unit 563 such as a hard disk drive or an SSD. A computer program for executing the learning process and the determination process described later is installed in the storage unit 563. Also, the information processing unit 56 is communicably connected to the operation panel 51, the display unit 52, the camera 53, the receiving conveyor 20, and the shuttle conveyor 30 described above.

[0046] 2 is a block diagram conceptually showing the functions realized in the information processing unit 56. As shown in FIG. 2, the information processing unit 56 has a learning unit 61, an estimation unit 62, a determination unit 63, and a control unit 64. The functions of these units are realized by the computer serving as the information processing unit 56 operating in accordance with a computer program stored in a storage unit 563.

[0047] The learning unit 61 is a processing unit that generates a learning model M for estimating the aggregate type based on an image of the aggregate 9. The learning unit 61 performs machine learning using images of sample aggregate 9 in a wet state prepared for learning (hereinafter referred to as "learning images") and the known aggregate type of the sample aggregate 9 as training data. A convolutional neural network, which is a type of deep learning, is suitable as a machine learning algorithm. In an experiment conducted by the present inventors, when a convolutional neural network having a pooling layer that performs image blurring processing was used as the learning unit 61, a learning model M with sufficient estimation accuracy could be generated even for images that were blurred in the conveying direction.

[0048] However, the machine learning algorithm used by the learning unit 61 is not limited to a convolutional neural network, and may be other supervised machine learning algorithms such as a neural network, linear regression, decision tree, or support vector regression.

[0049] The learning unit 61 learns the relationship between a large number of images of sample aggregates 9 and the aggregate types of the sample aggregates 9. As a result, a learning model M is generated that can output an estimation result of the aggregate type corresponding to an image of wet aggregate 9. The learning unit 61 provides the generated learning model M to the estimation unit 62. Details of the learning process will be described later.

[0050] The estimation unit 62 is a processing unit that estimates the aggregate type from the image of the aggregate 9 using the above learning model M. The estimation unit 62 extracts an image from the video D captured by the camera 53 and inputs it to the learning model M. Then, the learning model M outputs an estimation result E of the aggregate type based on the input image of the aggregate 9. The estimation unit 62 sends the estimation result E output from the learning model M to the determination unit 63.

[0051] The determination unit 63 acquires the aggregate type information I input to the operation panel 51. Also, the determination unit 63 acquires the estimation result E of the aggregate type by the estimation unit 62. Then, the determination unit 63 determines whether the aggregate type information I input to the operation panel 51 and the estimation result E of the aggregate type by the estimation unit 62 match. Further, the determination unit 63 outputs the determination result R to the display unit 52. As a result, on the display unit 52, the determination result R as to whether the aggregate type of the aggregate 9 conveyed by the receiving conveyor 20 matches the aggregate type specified by the operation panel 51 is displayed.

[0052] The control unit 64 controls the operation of the receiving conveyor 20 according to the determination result R of the determination unit 63. When the determination unit 63 determines that the aggregate type information I input to the operation panel 51 and the estimation result E of the estimation unit 62 match, the control unit 64 continues the operation of the receiving conveyor 20. When the determination unit 63 determines that the aggregate type information I input to the operation panel 51 and the estimation result E of the estimation unit 62 do not match, the control unit 64 stops the operation of the receiving conveyor 20.

[0053] <1-3. About the learning process> Subsequently, the learning process executed in the above-described aggregate determination device 50 will be described. FIG. 3 is a flowchart showing the flow of the learning process. This learning process is executed in advance before the determination process of the aggregate 9 described later.

[0054] When performing the learning process, first, prepare the learning sample aggregate 9 (step S11). For example, when it is desired to learn four aggregate types A to D, prepare the sample aggregates 9 of the four aggregate types. Then, sprinkle water on the prepared sample aggregate 9 (step S12). Thereby, the surface of the sample aggregate 9 is made uniformly wet. The color of the surface of the aggregate 9 changes between the dry aggregate 9 and the wet aggregate 9. In step S12, by sprinkling water on the sample aggregate 9, the variation in the color of the surface of the sample aggregate 9 is suppressed.

[0055] Next, the operator sequentially inputs the wet sample aggregates 9 one by one into the receiving conveyor 20 and operates the receiving conveyor 20. Then, while irradiating light from the light source 54, the camera 53 captures a moving image D of the sample aggregate 9 (step S13). When the capturing of the sample aggregate 9 of one aggregate type is completed, the operator inputs the sample aggregate 9 of the next aggregate type into the receiving conveyor 20. Then, the receiving conveyor 20 is operated, and in the same manner as above, the moving image D of the sample aggregate 9 is captured. In this way, the moving images D of all the prepared sample aggregates 9 of all aggregate types are sequentially captured.

[0056] The moving image D captured by the camera 53 is composed of a large number of frame images. For example, when shooting for 5 minutes with a camera 53 having a frame rate of 30 frames / second, the moving image D obtained by this shooting is composed of 9000 frame images. In step S13, for all the prepared aggregate types, such a moving image D (an aggregate of frame images) is captured. Then, the moving image D obtained by the shooting is transmitted from the camera 53 to the information processing unit 56.

[0057] Inside the information processing unit 56, a large number of frame images included in the video D are accumulated as learning images in the memory unit 563 (step S14). As described above, the video D includes a large number of frame images. Therefore, the learning unit 61 can prepare a large number of learning images. FIG. 4 is a diagram showing an example of frame images included in the video D. As shown in FIG. 4, due to the relationship between the frame rate of the camera 53 and the moving speed of the belt 23, the images have blurring in the conveying direction.

[0058] The learning unit 61 classifies the numerous learning images into teacher images and validation images (step S15). For example, if 36,000 learning images are obtained by photographing four types of sample aggregates 9 for five minutes each, three-quarters of the learning images, or 27,000, are used as teacher images, and the remaining one-quarter, or 9,000, are used as validation images. However, the ratio of teacher images to validation images does not necessarily have to be 3:1.

[0059] The learning unit 61 sets a set of a training image and the known aggregate type of the sample aggregate 9 shown in the training image as training data. The learning unit 61 also sets a set of a validation image and the known aggregate type of the sample aggregate 9 shown in the validation image as validation data.

[0060] The learning unit 61 performs image processing such as blurring, left-right flipping, and up-down flipping on the teacher image (step S16). By varying the teacher image through image processing, the range of images to be learned can be expanded. This can improve the accuracy of machine learning. However, it is desirable that the image processing does not include image rotation processing. As mentioned above, the teacher image is an image that has blur in the transport direction, and if rotation processing is performed, the direction of this blur will change, which can actually reduce the accuracy of machine learning. Note that the large number of teacher images may include images that have not been subjected to image processing.

[0061] The learning unit 61 performs machine learning using a large number of teacher data after image processing (step S17). That is, the machine learning algorithm inputs a large number of teacher images one by one into the learning model M. Then, the parameters of the learning model M are adjusted so that the estimated result of the aggregate type output from the learning model M approaches the aggregate type of the input teacher image. The machine learning algorithm repeats such parameter adjustment processing for the number of prepared teacher data.

[0062] After that, the machine learning algorithm uses a plurality of validation images to confirm the estimation accuracy of the aggregate type by the learning model M (step S18). Specifically, a validation image is input into the learning model M, and it is confirmed whether the estimated result of the aggregate type output from the learning model M matches the aggregate type of the input validation image. When the estimation accuracy of the aggregate type reaches a predetermined level (for example, a correct answer rate of 95% or more), the machine learning algorithm ends the learning process. As a result, one learning model M capable of outputting an estimated result of the aggregate type based on the image of the wet aggregate 9 is generated.

[0063] Note that if the estimation accuracy of the aggregate type does not reach the desired level, the above-described processing of steps S12 to S18 may be repeated again. Also, after generating the learning model M, an image of the wet state of the sample aggregate 9 different from the above-described teacher image and validation image may be input into the learning model M, and it may be further confirmed whether the estimated result of the aggregate type output from the learning model M matches the aggregate type of the sample aggregate 9 shown in the image.

[0064] Even for the same aggregate type, the surface color of the aggregate 9 may differ depending on the crushing location. Therefore, in step S13 described above, for each aggregate type, a video D of the sample aggregate 9 with different crushing locations may be acquired by shooting multiple times.

[0065] Furthermore, images of sample aggregates 9 collected at different crushing locations may be included among the many learning images input to the learning model M. For example, when 9,000 learning images are used for one aggregate type, images of multiple sample aggregates 9 collected at different crushing locations may be mixed among the 9,000 images. This allows the learning unit 61 to learn images of sample aggregates 9 collected at multiple crushing locations for each aggregate type. This further improves the estimation accuracy of the aggregate type output from the learning model M.

[0066] <1-4. Aggregate determination process> Next, a process for determining the suitability of the aggregate 9 put into the receiving conveyor 20 using the learning model M when the aggregate 9 is actually received will be described. Fig. 5 is a flowchart showing the flow of the determination process.

[0067] When receiving aggregates 9, first, the aggregates 9 loaded on the truck 90 are dumped into the receiving hopper 10 (step S21). Next, the operator inputs information I about the aggregate type of the dumped aggregates 9 into the operation panel 51 (step S22). For example, the operator inputs the aggregate type information I by pressing key 511 on the operation panel 51. Thereafter, the operator presses key 512 on the operation panel 51 to instruct the start of conveyance. This starts the operation of the receiving conveyor 20 (step S23).

[0068] The aggregates 9 supplied from the receiving hopper 10 to the upper surface of the belt 23 are transported obliquely upward toward the terminal pulley 22 as the belt 23 rotates. At this time, the water spray nozzles 58 spray water onto the upper surface of the belt 23. The aggregates 9 transported by the receiving conveyor 20 are sprayed with water as they pass below the water spray nozzles 58 (step S24). This causes the surfaces of the aggregates 9 to become wet.

[0069] Thereafter, the wet aggregate 9 continues to be conveyed by the receiving conveyor 20. When the aggregate 9 passes below the camera 53, the camera 53 captures a moving image D of the aggregate 9 (step S25). Thereby, the moving image D of the wet aggregate 9 is obtained. The camera 53 transmits the obtained moving image D to the information processing unit 56.

[0070] Inside the information processing unit 56, the estimation unit 62 cuts out one image (frame image) from the above-mentioned moving image D (step S26). Then, the estimation unit 62 inputs the cut-out image to the learning model M. Then, the learning model M outputs an estimation result E of the aggregate type based on the input image of the aggregate 9 (step S27). The estimation unit 62 sends the estimation result E output from the learning model M to the determination unit 63.

[0071] The determination unit 63 acquires the aggregate type information I input to the operation panel 51 in step S22 above. Also, the determination unit 63 acquires the estimation result E output from the learning model M in step S27 above. Then, the determination unit 63 determines whether or not the aggregate type information I input to the operation panel 51 matches the estimation result E of the aggregate type by the estimation unit 62 (step S28).

[0072] The determination unit 63 outputs the determination result R to the display unit 52. Thereby, on the display unit 52, the determination result R as to whether or not the aggregate type of the aggregate 9 conveyed by the receiving conveyor 20 matches the aggregate type specified by the operation panel 51 is displayed (step S29). The operator can recognize the determination result R by checking the display unit 52.

[0073] The control unit 64 checks whether the determination result R of the determination unit 63 in the above step S28 is either "match" or "mismatch" (step S30). Then, when the determination result R indicates that the aggregate type information I input to the operation panel 51 matches the estimation result E of the estimation unit 62 (step S23: Yes), the control unit 64 continues the operation of the receiving conveyor 20. On the other hand, when the determination result R indicates that the aggregate type information I input to the operation panel 51 does not match the estimation result E of the estimation unit 62 (step S30: No), the control unit 64 stops the operation of the receiving conveyor 20 (step S31).

[0074] When the determination result R is "mismatch", there is a possibility that the aggregate 9 of the inappropriate aggregate type has been put into the receiving hopper 10, or that the aggregate type information I has been incorrectly input to the operation panel 51. In either case, the aggregate 9 on the receiving hopper 10 may be conveyed to the wrong silo 40. In step S31 above, in such a case, the operation of the receiving conveyor 20 is urgently stopped. This stops the conveyance of the aggregate 9 and prevents the aggregate 9 from being conveyed to the wrong silo 40.

[0075] As described above, this aggregate determination device 50 photographs the aggregate 9 conveyed by the receiving conveyor 20, and uses the obtained image and the learning model M generated in advance by machine learning to estimate the aggregate type of the aggregate 9 during conveyance. Then, the aggregate determination device 50 determines whether the estimation result E of the aggregate type matches the aggregate type information I input to the operation panel 51. Thereby, it is possible to detect that the aggregate 9 may be conveyed to the wrong silo 40.

[0076] In particular, in this aggregate determination device 50, regardless of whether the surface of the aggregate 9 fed onto the conveyor is in a dry state or a wet state, water is sprayed onto the aggregate 9 to uniformly wet the surface of the aggregate 9. Then, based on the image of the wet aggregate 9, the aggregate type is estimated. As a result, variations in the surface state of the aggregate 9 can be suppressed, and the aggregate type can be accurately estimated. Therefore, it is possible to accurately determine whether the aggregate 9 fed onto the receiving conveyor 20 matches the aggregate type information I input to the operation panel 51.

[0077] Also, by wetting the surface of the aggregate 9 by spraying water, the scattering of dust can be suppressed. As a result, it is possible to prevent dust from adhering to the lens of the camera 53. Therefore, it is possible to suppress the occurrence of fogging due to dust in the video D captured by the camera 53. For this reason, the aggregate determination device 50 can output a stable determination result R over a long period of time.

[0078] Further, in the aggregate determination device 50 of the present embodiment, the aggregate 9 is photographed without stopping the receiving conveyor 20. For this reason, the image input to the learning model M is not a high-definition image but an image with blur. Instead of a high-speed camera with a high frame rate capable of acquiring a frame image without blur in the transport direction or a high-performance camera with a large number of pixels, an inexpensive camera that is easy to obtain can be used for the camera 53 that photographs the aggregate 9. Thereby, the cost of the camera 53 can be reduced. Also, by using an inexpensive camera 53 that is easy to obtain, the replacement cost when the camera 53 fails can also be suppressed. Therefore, it is possible to realize an aggregate determination device 50 with excellent maintainability.

[0079] Furthermore, in the aggregate discrimination device 50 of this embodiment, a convolutional neural network that performs image blurring processing is used to generate the learning model M in the learning unit 61. Therefore, the image input to the learning model M does not necessarily need to be a high-resolution image. That is, even if an image is blurred in the conveyance direction, a learning model M with high estimation accuracy can be obtained. Therefore, a learning model M with high estimation accuracy can be obtained using an inexpensive camera 53 instead of a high-performance camera. The camera 53 may be, for example, a webcam with a frame rate of approximately 30 frames per second. Furthermore, by reducing the frame rate and the number of pixels, the memory capacity of the aggregate discrimination device 50 can be reduced. That is, an aggregate discrimination device with sufficient estimation accuracy can be realized despite its inexpensive configuration.

[0080] If the frame rate of the camera 53 is 30 frames per second, frame images with blur in the conveyance direction can be acquired by setting the moving speed of the belt 23 of the receiving conveyor 20 to 50 m / min or more (moving distance per frame: 2.78 cm / frame). Furthermore, if the moving speed of the belt 23 exceeds 120 m / min (moving distance per frame: 6.67 cm / frame), the frame images obtained by shooting will be blurred to the extent that the outline of the aggregate cannot be discerned. Therefore, it is recommended to adjust the relationship between the frame rate of the camera 53 and the moving speed of the belt 23 so that the moving distance per frame is in the range of 2.78 cm / frame or more and 6.67 cm / frame or less.

[0081] Furthermore, in the aggregate determination device 50 of this embodiment, an image is captured by the camera 53 while irradiating light from the light source 54 inside the hood 55. This makes it possible to reduce variations in brightness of the image input to the learning model M. Therefore, the aggregate type can be estimated with higher accuracy.

[0082] The valve 581 of the water spray nozzle 58 may be electrically connected to the information processing unit 56 so that the valve 581 is opened and closed in accordance with the operation of the receiving conveyor 20 .

[0083] <2. Second Embodiment (Moisture Sensor)> Next, the second embodiment of the present invention will be described. FIG. 6 is a diagram showing the configuration of the aggregate conveying device 1 according to the second embodiment.

[0084] This embodiment is different from the above-described first embodiment in that the aggregate determination device 50 includes a moisture sensor 57. The moisture sensor 57 is a sensor that measures the moisture content on the surface of the aggregate 9 conveyed by the receiving conveyor 20. The moisture sensor 57 is disposed at a position above the receiving conveyor 20, downstream of the water spray nozzle 58, and upstream of the camera 53. Further, the moisture sensor 57 is disposed in the hood 55 together with the camera 53 and the light source 54.

[0085] For example, a near-infrared moisture meter is used as the moisture sensor 57. The near-infrared moisture meter measures the moisture content of the aggregate 9 by irradiating the aggregate 9 with near-infrared rays and measuring the absorbance on the surface of the aggregate 9 based on the amount of reflected light. However, the moisture sensor 57 may be a moisture meter of another type. The moisture sensor 57 is communicably connected to the information processing unit 56. The measured value of the moisture sensor 57 is transmitted to the information processing unit 56.

[0086] FIG. 7 is a flowchart showing the flow of the determination process in the second embodiment. As shown in FIG. 7, in this second embodiment, the processes of steps S21 to S24 are performed in the same manner as in the first embodiment. After water is sprayed from the water spray nozzle 58 onto the aggregate 9, the moisture sensor 57 measures the moisture content on the surface of the aggregate 9 (step S24A). Then, the moisture sensor 57 transmits the obtained measured value of the moisture content to the information processing unit 56.

[0087] In this embodiment, the valve 581 of the water spray nozzle 58 is electrically connected to the information processing unit 56. The control unit 64 of the information processing unit 56 switches the opening degree of the valve 581 according to the measured value input from the moisture sensor 57. Thereby, the amount of water sprayed from the water spray nozzle 58 is switched (step S24B).

[0088] For example, if the measurement value of moisture sensor 57 is lower than a preset threshold value, control unit 64 increases the opening of valve 581 to increase the amount of water sprayed from watering nozzle 58. On the other hand, if the measurement value of moisture sensor 57 is higher than the preset threshold value, control unit 64 decreases the opening of valve 581 to decrease the amount of water sprayed from watering nozzle 58.

[0089] Thereafter, the aggregate determination device 50 performs the processes of steps S25 to S31 in the same manner as in the first embodiment.

[0090] The aggregate 9 fed into the receiving hopper 10 may be dry or already wet, depending on the environmental conditions of the day, such as the weather, humidity, and temperature. However, in this embodiment, the amount of water sprayed from the water spray nozzle 58 is adjusted according to the amount of moisture on the surface of the aggregate 9 measured by the moisture sensor 57. This makes it possible to keep the amount of moisture on the surface of the aggregate 9 photographed by the camera 53 approximately constant, regardless of the environmental conditions, such as the weather, humidity, and temperature. This allows the type of aggregate to be estimated more accurately in step S27.

[0091] Moreover, if the aggregate 9 is overly wet, problems arise such as water accumulating at the bottom of the silo 40 and the moisture content being unstable when producing ready-mixed concrete. As in the present embodiment, by adjusting the moisture content on the surface of the aggregate 9 in the receiving conveyor 20, the moisture content of the aggregate 9 stored in the silo 40 can be made approximately constant. Therefore, excess moisture can be prevented from accumulating in the silo 40.

[0092] Note that control unit 64 may switch between opening and closing valve 581 in accordance with the measurement value of moisture sensor 57, thereby switching between the presence and absence of water spray from water spray nozzle 58.

[0093] In the example of FIG. 6, the moisture sensor 57 is disposed on the downstream side of the conveying path with respect to the watering nozzle 58. However, the moisture sensor 57 may be disposed on the upstream side of the conveying path with respect to the watering nozzle 58. However, if the moisture sensor 57 is disposed on the downstream side of the conveying path with respect to the watering nozzle 58, the measured value of the moisture sensor 57 will more accurately indicate the moisture content of the aggregate 9 at the time of imaging by the camera 53. Therefore, it is easy to manage the measured value of the moisture sensor 57 and the image obtained by the camera 53 in association with each other.

[0094] Further, the measured value of the moisture sensor 57 may be included in the data input to the learning model M. For example, even during the prior learning process, the moisture content is measured by the moisture sensor 57, and a set of the measured value and the image is used as input data to generate the learning model M. Then, at the time of actually receiving the aggregate 9, a set of the measured value of the moisture sensor 57 and the image of the aggregate 9 is input to the learning model M to output the estimation result E of the aggregate type. In this way, the aggregate type can be estimated with higher accuracy in consideration of the moisture content on the surface of the aggregate 9.

[0095] <3. Third Embodiment (Majority Voting)> Subsequently, a third embodiment of the present invention will be described. FIG. 8 is a flowchart showing the flow of the determination process in the third embodiment. As shown in FIG. 8, in this third embodiment, similar to the first embodiment, the processes of steps S21 to S25 are performed to obtain the moving image D of the aggregate 9.

[0096] The estimation unit 62 cuts out an odd number (an odd number of 3 or more) of images from the moving image D (step S26). Then, the estimation unit 62 inputs each of the cut-out odd-numbered images to the learning model M. Then, the learning model M outputs an odd number of estimation results E corresponding to each of the input odd-numbered images of the aggregate 9 (step S27). The estimation unit 62 sends the odd number of estimation results E output from the learning model M to the determination unit 63.

[0097] Next, the determination unit 63 compares the odd number of estimation results E to determine whether they each match the aggregate type information I input to the operation panel 51. Then, the determination unit 63 makes a final determination by majority vote of the comparison results (step S28). That is, if there are more "matches" than "mismatches" among the odd number of comparison results, the determination unit 63 determines the final determination result R to be "matches." On the other hand, if there are more "mismatches" than "matches" among the odd number of comparison results, the determination unit 63 determines the final determination result R to be "mismatches."

[0098] Thereafter, the aggregate determination device 50 performs the processes of steps S29 to S31 in the same manner as in the first embodiment.

[0099] As described above, in the third embodiment, an odd number of estimation results E are output from the learning model M based on an odd number of images of aggregates 9. Then, based on the odd number of estimation results E, it is determined by majority vote whether the aggregate 9 fed onto the receiving conveyor 20 matches the aggregate type input to the operation panel 51. This makes it possible to obtain a more accurate determination result R.

[0100] The aggregate determination device 50 may have an odd number of cameras 53, three or more. An odd number of images may be acquired by cutting out one image from each of the videos D captured by the odd number of cameras 53.

[0101] <4. Fourth embodiment (additional learning)> Next, a fourth embodiment of the present invention will be described. This fourth embodiment differs from the first embodiment in that additional machine learning is performed as appropriate while performing the above-mentioned determination process using a learning model M that has been generated. Figure 9 is a flowchart showing the flow of processing when additional machine learning is performed.

[0102] 9, after the judgment result R is displayed on the display unit 52, the operator determines whether or not the displayed judgment result R is correct. Then, the operator inputs correct / incorrect information indicating whether or not the judgment result R is correct into the operation panel 51 (step S32).

[0103] For example, when the judgment result R of "mismatch" is displayed on the display unit 52, the operator checks whether or not there is indeed a mismatch between the aggregate type of the aggregate 9 fed onto the receiving conveyor 20 and the aggregate type information I input into the operation panel 51. If there is indeed a mismatch, the operator inputs into the operation panel 51 that the judgment result R by the judgment unit 63 is correct. On the other hand, when the aggregate type of the aggregate 9 fed onto the receiving conveyor 20 matches the aggregate type information I input into the operation panel 51, the operator inputs into the operation panel 51 that the judgment result R by the judgment unit 63 is incorrect.

[0104] The correct / incorrect information input to the operation panel 51 is transmitted to the information processing unit 56. The information processing unit 56 stores a set of the image cut out in step S26, the estimation result E in step S27, and the correct / incorrect information input in step S32 in the memory unit 563 as training data for additional learning (step S33). The information processing unit 56 stores such training data every time the receiving conveyor 20 receives aggregate 9.

[0105] In step S33, not only the images (frame images) extracted in step S26 but also a video D of a predetermined time (e.g., 5 minutes) including the images may be included in the training data for additional learning. In this way, many frame images included in the video D can be used as images for additional learning. Therefore, a large amount of training data can be accumulated in a short period of time.

[0106] Thereafter, the learning unit 61 of the information processing unit 56 performs additional machine learning using the accumulated training data at a predetermined timing (step S34). The timing for performing additional machine learning may be, for example, when the rate at which the above-mentioned true / false information is "incorrect" becomes higher than a predetermined threshold. The learning unit 61 updates or recreates the learning model M using only the training data for additional learning, or the initial training data plus the training data for additional learning. This allows the estimation accuracy of the aggregate type by the learning model M to be maintained at a predetermined level.

[0107] <5. Variations> Although the first to fourth embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments.

[0108] In the above embodiment, the learning unit 61 performs image processing on the frame images, such as blurring, left-right inversion, and up-down inversion. However, the frame images may be used as learning images as they are without performing any image processing.

[0109] Furthermore, in the above embodiment, the camera 53 captures the video D of the aggregates 9 being transported by the receiving conveyor 20. However, the camera 53 may also capture a still image of the aggregates 9 being transported by the receiving conveyor 20. Even when capturing a still image instead of the video D, it is possible to obtain an image that includes blur by capturing the image without stopping the transport of the aggregates 9. In other words, the camera 53 is not limited to capturing the video D, and may also capture a still image of the aggregates 9 being transported to obtain an image that includes blur as a learning image.

[0110] In the above embodiment, the receiving conveyor 20 is stopped when the determination result R by the determination unit 63 is "mismatch." However, for example, a cut gate may be provided below the receiving hopper 10, and the cut gate may be closed when the determination result R is "mismatch." A surge bin for temporarily storing the aggregate 9 may be provided on the conveyance path, and the aggregate 9 may be conveyed to the surge bin when the determination result R is "mismatch." A gate may be provided somewhere on the conveyance path to the silo 40, and the gate may be closed when the determination result R is "mismatch." That is, the control unit 64 may restrict the conveyance of the aggregate 9 to the silo 40 in some way when the determination result R is "mismatch."

[0111] In the above embodiment, the receiving conveyor 20 is inclined. However, the receiving conveyor 20 does not have to be inclined. In other words, the receiving conveyor 20 may transport the aggregates 9 in a horizontal direction.

[0112] In the above embodiment, the computer serving as the information processing unit 56 is located within the concrete plant. However, some or all of the functions of the information processing unit 56 may be located outside the concrete plant. For example, the computer serving as the information processing unit 56 may be located in a remote location and may be able to communicate with the operation panel 51, the display unit 52, and the camera 53 via the Internet.

[0113] In the above embodiment, the determination result R of the determination unit 63 is displayed on the display unit 52. However, when the determination result R of "mismatch" is output from the determination unit 63, a warning lamp may be turned on or a warning buzzer may be sounded.

[0114] In the above-described embodiment, an example of determining the type of aggregate was explained in the receiving conveyor 20 of the concrete plant. However, the present invention can also be used in a quarry that ships and supplies aggregate 9 to the concrete plant. That is, the aggregate determination device of the present invention may determine the type of aggregate in a conveyor installed at the quarry.

[0115] Also, the respective elements that appeared in the above-described embodiment and modification examples may be appropriately combined within a range where no contradiction occurs.

Industrial Applicability

[0116] The present invention can be used to determine the type of aggregate in a concrete plant or a quarry.

Explanation of Signs

[0117] 1 Aggregate conveying device 9 Aggregate 10 Receiving hopper 20 Receiving conveyor 30 Shuttle conveyor 40 Silo 50 Aggregate determination device 51 Operation panel 52 Display unit 53 Camera 54 Light source 55 Hood 56 Information processing unit 57 Moisture sensor 58 Water spray nozzle 61 Learning unit 62 Estimation unit 63 Judgment unit 64 Control unit 90 Truck D Video E Estimation result I Information M Learning model R Judgment result

Claims

1. An aggregate determination device for determining the suitability of aggregates fed onto a conveyor, comprising: a water spray nozzle for spraying water on the aggregates; a camera for photographing the aggregates conveyed by the conveyor, located downstream of the water spray nozzle in the conveying path; an operation panel into which information on the type of aggregate is input; an information processing unit communicably connected to the camera and the operation panel; wherein the information processing unit includes: an estimation unit that inputs an image of the aggregates obtained by photographing with the camera into a learning model generated by machine learning and outputs an estimated result of the aggregate type from the learning model; a determination unit that determines whether or not the aggregate type input to the operation panel matches the estimated result; a moisture sensor for measuring the moisture content on the surface of the aggregates conveyed by the conveyor; a control unit that switches the presence or absence of water spraying or the amount of water spraying from the water spray nozzle according to the measured value of the moisture sensor. The aggregate determination device further comprising the above components.

2. The aggregate determination device according to Claim 1, wherein the moisture sensor is located downstream of the water spray nozzle in the conveying path.

3. The aggregate determination device according to Claim 1 or Claim 2, wherein the camera photographs a moving image of the aggregates conveyed by the conveyor, the moving image includes a plurality of frame images having blurring in the conveying direction of the aggregates, and the information processing unit inputs the frame images included in the moving image photographed by the camera into the learning model.

4. The aggregate determination device according to Claim 3, wherein the information processing unit further includes a learning unit that generates the learning model by performing machine learning using, as teacher data, a plurality of frame images included in a moving image of wet sample aggregates prepared for learning and the known aggregate type of the sample aggregates.

5. The aggregate determination device according to Claim 4, wherein the operation panel can input correct / incorrect information indicating whether or not the determination result by the determination unit is correct, and the learning unit updates the learning model by performing additional machine learning using, as teacher data, the image of the aggregates obtained from the camera, the estimated result, and the correct / incorrect information.

6. The aggregate determination device according to any one of Claims 1 to 5. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ The estimation unit inputs the odd-numbered images obtained from the camera into the learning model, and outputs an odd number of estimation results corresponding to each of the odd-numbered images from the learning model. The determination unit compares the aggregate type input to the operation panel with the odd number of estimation results, and performs determination by a majority vote of the comparison results. An aggregate determination device.

7. An aggregate determination method for determining the suitability of aggregates fed onto a conveyor, comprising: a) a step of inputting information on the aggregate type of the aggregates fed onto the conveyor into an operation panel; b) a step of sprinkling water on the aggregates; c) after the step b), a step of photographing the aggregates conveyed by the conveyor; d) a step of inputting the images of the aggregates obtained by the photographing in the step c) into a learning model generated by machine learning, and outputting an estimation result of the aggregate type from the learning model; e) a step of determining whether or not the aggregate type input in the step a) matches the estimation result; and measuring the moisture content on the surface of the aggregates conveyed by the conveyor, and switching the presence or absence or the amount of water sprinkling in the step b) according to the obtained measurement value. An aggregate determination method.

Images