Aggregate determination device and aggregate determination method

Abstract

To provide an aggregate determination device and an aggregate determination method that can determine whether an aggregate introduced to a conveyor matches aggregate types input to a control panel.SOLUTION: An aggregate determination device 50 comprises a control panel 51, a camera 53, and an information processing unit 56. The camera 53 photographs an aggregate 9 conveyed by a reception conveyor 20. An operator inputs, to the control panel 51, information on the aggregate type of the aggregate 9 introduced to the reception conveyor 20. The information processing unit 56 inputs an image of the aggregate 9 obtained through the photographing performed by the camera 53 to a learning model generated through machine learning. Consequently, a result of estimation of the aggregate type is output from the learning model. The information processing unit 56 determines whether the information on the aggregate type input to the control panel 51 and the estimation result output from the learning model match each other.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an aggregate determination device and an aggregate determination method for determining whether aggregate fed onto a conveyor is suitable. [Background technology]

[0002] Conventionally, concrete plants are provided with multiple silos for storing aggregates, which are raw materials for concrete. Aggregates transported by truck from a quarry are transported by conveyors within the concrete plant and stored in designated silos for each type of aggregate. A conventional concrete plant is described, for example, in Patent Document 1. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 08-091580 Summary of the Invention [Problem to be solved by the invention]

[0004] Aggregates are classified into several types according to characteristics such as particle size, color, and place of origin. In concrete plants, a specific silo is designated for storing each type of aggregate. When aggregate is dumped from a truck onto a conveyor, the truck driver or a designated worker inputs the aggregate type into a control panel. This sets the destination of the aggregate by the conveyor to the silo corresponding to that aggregate type.

[0005] If the wrong aggregate is fed onto the conveyor, or if the wrong type of aggregate is entered into the control panel, the fed aggregate will be transported to the wrong silo. This will result in the silo being mixed with aggregate of a different type. In this case, it will be impossible to properly control the quality of the ready-mix concrete produced, and it will be necessary to remove all of the aggregate from the silo where the mixing occurred. This will result in a great deal of work and damage.

[0006] The present invention has been developed in consideration of the above circumstances, and aims to provide an aggregate determination device and an aggregate determination method that can determine whether the aggregate type fed into a conveyor matches the aggregate type input into an operation panel. [Means for solving the problem]

[0007] In order to solve the above problems, the first invention of the present application is an aggregate judgment device that judges the suitability of aggregate fed onto a conveyor, and includes a camera that photographs the aggregate transported by the conveyor, an operation panel into which information on the type of aggregate is input, and an information processing unit that is communicatively connected to the camera and the operation panel.The information processing unit has an estimation unit that inputs images of the aggregate obtained by the camera into a learning model generated by machine learning and outputs an estimation result of the aggregate type from the learning model, and a judgment unit that judges whether the aggregate type input into the operation panel matches the estimation result.

[0008] The second invention of the present application is an aggregate determination device of the first invention, wherein the camera captures a video of the aggregate being transported by the conveyor, the video including a plurality of frame images having a blur in the transport direction of the aggregate, and the information processing unit inputs the frame images included in the video captured by the camera into the learning model.

[0009] A third invention of the present application is the aggregate determination device of the second invention, wherein the information processing unit further has a learning unit that generates the learning model by performing machine learning using a plurality of frame images included in a video of sample aggregate prepared for learning and the known aggregate type of the sample aggregate as training data.

[0010] A fourth aspect of the present invention is the aggregate determination device of the third aspect, wherein the learning unit performs the machine learning after performing image processing other than rotation on the frame images.

[0011] The fifth invention of the present application is an aggregate judgment device of the third or fourth invention, wherein the operation panel is capable of inputting correct / incorrect information indicating whether the judgment result by the judgment unit is correct or not, and the learning unit updates the learning model by performing additional machine learning using the image of the aggregate obtained from the camera, the estimation result, and the correct / incorrect information as training data.

[0012] A sixth aspect of the present invention is the aggregate determination device according to any one of the first to fifth aspects of the present invention, further comprising a light source that irradiates light toward the imaging range of the camera.

[0013] A seventh aspect of the present invention is the aggregate determination device of the sixth aspect, further comprising a hood that covers a portion of the upper surface of the conveyor, and the camera and the light source are disposed within the hood.

[0014] An eighth aspect of the present invention is the aggregate determination device according to any one of the first to seventh aspects of the present invention, further comprising a result output unit that outputs the determination result of the determination unit.

[0015] A ninth invention of the present application is an aggregate determination device according to any one of the first to eighth inventions, further comprising a moisture sensor that measures the moisture content on the surface of aggregate transported by the conveyor, wherein the information processing unit has a first learning model generated by performing the machine learning on aggregate whose surface moisture content is less than a predetermined value, and a second learning model generated by performing the machine learning on aggregate whose surface moisture content is equal to or greater than the predetermined value, and wherein the estimation unit, when the moisture content measured by the moisture sensor is less than the predetermined value, inputs an image of the aggregate taken by the camera to the first learning model and outputs an estimation result of the aggregate type from the first learning model, and when the moisture content measured by the moisture sensor is equal to or greater than the predetermined value, inputs the image of the aggregate obtained by the camera to the second learning model and outputs an estimation result of the aggregate type from the second learning model.

[0016] The tenth invention of the present application is an aggregate determination device according to any one of the first to ninth inventions, wherein the estimation unit inputs an odd number of images obtained from the camera into the learning model and outputs an odd number of estimation results corresponding to each of the odd number of images from the learning model, and the determination unit compares the aggregate type input to the control panel with the odd number of estimation results and makes a determination by majority vote of the comparison results.

[0017] The 11th invention of the present application is an aggregate determination device of any one of the 1st to 10th inventions, wherein the information processing unit further has a control unit that restricts the transportation of aggregate to the silo when the determination unit determines that the aggregate type input to the control panel does not match the estimated result.

[0018] The twelfth invention of the present application is an aggregate determination method for determining the suitability of aggregate fed onto a conveyor, comprising the steps of: a) inputting information on the aggregate type of the aggregate fed onto the conveyor into an operation panel; b) photographing the aggregate being transported by the conveyor; c) inputting the image of the aggregate obtained by photographing in step b) into a learning model generated by machine learning, and outputting an estimation result of the aggregate type from the learning model; and d) determining whether the aggregate type input in step a) matches the estimation result. [Effects of the Invention]

[0019] According to the first to twelfth aspects of the present invention, it is possible to determine whether the type of aggregate fed onto the conveyor matches the type of aggregate input into the operation panel.

[0020] In particular, according to the second aspect of the present invention, a video of the aggregate is captured, and frame images with blur in the conveying direction are input to the learning model. The camera used to capture the aggregate does not need to be a high-speed camera with a high frame rate that can capture frame images without blur in the conveying direction, or a high-performance camera with a large number of pixels; instead, an easily available, inexpensive camera is sufficient. This reduces the cost of the camera. Furthermore, by using an easily available, inexpensive camera, the cost of replacing the camera when it breaks down can also be reduced. Therefore, an aggregate judgment device with excellent maintainability can be realized.

[0021] In particular, according to the third aspect of the present invention, a large number of frame images included in a video are used as learning images. Therefore, a large number of aggregate images required to generate a learning model can be acquired in a relatively short time. Furthermore, a learning model with sufficient estimation accuracy can be created using blurred frame images. Furthermore, the storage capacity of the aggregate determination device can be reduced compared to when a large number of high-resolution images are prepared. In other words, a learning model with high estimation accuracy can be generated with an inexpensive device configuration.

[0022] In particular, according to the fifth aspect of the present invention, the learning model can be updated by performing additional machine learning. This allows the estimation accuracy of aggregate types using the learning model to be maintained at a certain level. Furthermore, by storing images of aggregates, estimation results, and accuracy information, additional machine learning can be performed quickly.

[0023] In particular, according to the sixth aspect of the present invention, the variation in brightness is suppressed, thereby enabling the aggregate type to be estimated with higher accuracy.

[0024] In particular, according to the seventh aspect of the present invention, the type of aggregate can be estimated with higher accuracy by suppressing the influence of external ambient light.

[0025] In particular, according to the eighth aspect of the present invention, the user can recognize the determination result of the determination unit.

[0026] In particular, according to the ninth aspect of the present invention, different learning models are used when the moisture content of the surface of the aggregate is less than a predetermined value and when it is equal to or greater than the predetermined value, thereby enabling more accurate estimation of the aggregate type.

[0027] In particular, according to the tenth aspect of the present invention, more accurate estimation results can be obtained based on an odd number of images.

[0028] In particular, according to the eleventh aspect of the present invention, when there is a possibility that inappropriate aggregate has been added, the transport of the aggregate to the silo can be restricted. [Brief explanation of the drawings]

[0029] [Figure 1] 1 is a diagram showing a configuration of an aggregate transport device according to a first embodiment. [Figure 2] FIG. 2 is a block diagram conceptually showing functions realized in an information processing unit of the first embodiment. [Figure 3] 4 is a flowchart showing the flow of a learning process in the first embodiment. [Figure 4] FIG. 10 is a diagram showing an example of learning images extracted from a video. [Figure 5] 4 is a flowchart showing the flow of a determination process in the first embodiment. [Figure 6] FIG. 10 is a diagram showing the configuration of an aggregate transporting device according to a second embodiment. [Figure 7] 10 is a flowchart showing the flow of a learning process in the second embodiment. [Figure 8] 10 is a flowchart showing the flow of a determination process in the second embodiment. [Figure 9] 10 is a flowchart showing the flow of a determination process in the third embodiment. [Figure 10] 13 is a flowchart showing the flow of processing when additional machine learning is performed in the fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0030] Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

[0031] 1. First Embodiment <1-1. Configuration of aggregate transport device> 1 is a diagram showing the configuration of an aggregate conveying device 1 including an aggregate identification device 50. This aggregate conveying device 1 is a device that receives raw aggregate 9 in a concrete plant and conveys the aggregate 9 to multiple silos 40. As shown in FIG. 1, the aggregate conveying device 1 includes a receiving hopper 10, a receiving conveyor 20, a shuttle conveyor 30, four silos 40, and an aggregate identification device 50.

[0032] The receiving hopper 10 is positioned above the conveyance start position of the receiving conveyor 20. The receiving hopper 10 is a funnel-shaped hopper whose inner diameter gradually decreases as it goes downward. Aggregates 9 are transported from the quarry by truck 90 and dumped into the opening at the top of the receiving hopper 10. The dumped aggregates 9 are temporarily stored in the receiving hopper 10 and are then supplied to the top surface of the receiving conveyor 20 from the opening at the bottom of the receiving hopper 10.

[0033] The receiving conveyor 20 is a mechanism that transports the aggregates 9 placed in the receiving hopper 10 obliquely upward. The receiving conveyor 20 has a start pulley 21, a terminal pulley 22, and a circular belt 23 stretched between the pulleys 21 and 22. The start pulley 21 is located below the receiving hopper 10. The terminal pulley 22 is located above the shuttle conveyor 30, which will be described later. The terminal pulley 22 is located at a position spaced apart obliquely upward from the start pulley 21.

[0034] At least one of the start pulley 21 and the end pulley 22 is rotated by the power of a motor (not shown). As a result, a belt 23 rotates between the start pulley 21 and the end pulley 22 in the direction of the arrow in FIG. 1. The moving speed of the belt 23 is adjusted to any speed, for example, 50 m / min or higher. The aggregates 9 fed into the receiving hopper 10 are supplied to the upper surface of the belt 23 near the start pulley 21. Then, as the belt 23 rotates, the aggregates 9 placed on the belt 23 move obliquely upward from a position below the receiving hopper 10 toward the end pulley 22.

[0035] The belt 23 of the receiving conveyor 20 moves continuously at a constant speed. Therefore, the aggregates 9 drawn from the bottom of the receiving hopper 10 are transported while being placed at a substantially constant height on the upper surface of the belt 23. When the aggregates 9 are transported to the position of the terminal pulley 22, the belt 23 reverses and the aggregates 9 fall onto the shuttle conveyor 30.

[0036] The shuttle conveyor 30 is a mechanism for distributing the aggregates 9 transported from the receiving conveyor 20 to the four silos 40. The shuttle conveyor 30 is located below the terminal pulley 22 of the receiving conveyor 20. The shuttle conveyor 30 has a first pulley 31, a second pulley 32, and a circular belt 33 stretched between the pulleys 31 and 32.

[0037] At least one of the first pulley 31 and the second pulley 32 is rotated by the power of a motor (not shown). This causes the belt 33 to rotate between the first pulley 31 and the second pulley 32. The aggregates 9 that have fallen from the receiving conveyor 20 and placed on the belt 33 of the shuttle conveyor 30 are transported to one of the four silos 40 by the rotation of the belt 33.

[0038] The shuttle conveyor 30 is movable along a horizontal plane by a drive mechanism (not shown), which allows the destination of the aggregate 9 by the shuttle conveyor 30 to be switched to a specified one of the four silos 40.

[0039] The four silos 40 are storage tanks for aggregates 9. The aggregates 9 are classified into a plurality of aggregate types according to characteristics such as particle size, color, and place of origin. In this aggregate conveying device 1, a silo 40 to store each aggregate type is determined. In the example of FIG. 1, aggregates 9 of four aggregate types A to D are sorted and stored in the four silos 40. The number of silos 40 provided in the aggregate conveying device 1 is not limited to four.

[0040] <1-2. Aggregate detection device> The aggregate determination device 50 is a device for determining whether the aggregate 9 fed onto the receiving conveyor 20 is suitable. The aggregate determination device 50 photographs the aggregate 9 being transported by the receiving conveyor 20 and estimates the aggregate type based on the obtained image. The aggregate determination device 50 then determines whether the estimated aggregate type matches a specified aggregate type. As shown in FIG. 1, the aggregate determination device 50 includes an operation panel 51, a display unit 52, a camera 53, a light source 54, a hood 55, and an information processing unit 56.

[0041] The operation panel 51 is disposed near the receiving hopper 10. The operation panel 51 is electrically connected to an information processing unit 56. The operation panel 51 has a plurality of keys 511, 512. An operator (the driver of the truck 90 or a worker at the concrete plant) presses the key 511 indicating the type of aggregate 9 to be dumped into the receiving hopper 10. This causes information on the aggregate type to be input from the operation panel 51 to the information processing unit 56. Furthermore, when the information on the aggregate type is input, the shuttle conveyor 30 changes its position so that the silo 40 for that aggregate type becomes the destination.

[0042] After the aggregates 9 are loaded into the receiving hopper 10, the operator presses the key 512 to instruct the start of transport. This starts the operation of the receiving conveyor 20 and the shuttle conveyor 30. As a result, the loaded aggregates 9 are transported to the specified silo 40 by the receiving conveyor 20 and the shuttle conveyor 30.

[0043] The operation panel 51 may be configured with a general keyboard, mouse, or touch panel.

[0044] The display unit 52 is disposed near the receiving hopper 10 together with the operation panel 51. The display unit 52 is electrically connected to the information processing unit 56. For example, a liquid crystal display is used as the display unit 52. The display unit 52 displays various information related to the aggregate determination device 50. For example, the display unit 52 displays the determination results output from a determination unit 63 (described later) of the information processing unit 56. In other words, the display unit 52 is an example of the "result output unit" of the present invention.

[0045] The camera 53 is an imaging device that captures images of the aggregates 9 transported by the receiving conveyor 20. The camera 53 is arranged downstream of the receiving hopper 10 on the transport path so as to face the upper surface of the belt 23 of the receiving conveyor 20. In this embodiment, the camera 53 captures video of the aggregates 9 transported by the receiving conveyor 20. After the receiving conveyor 20 starts operating, a predetermined time has passed and the leading aggregate 9 has passed below the camera 53, and the camera 53 starts capturing video. The camera 53 then inputs the obtained video to the information processing unit 56.

[0046] A sensor for detecting aggregates 9 may be provided midway along the receiving conveyor 20, and the camera 53 may start taking video based on a detection signal from the sensor.

[0047] The light source 54 is a component for emitting light toward the shooting range of the camera 53. For example, an LED lamp is used as the light source 54. However, the light source 54 may also be a fluorescent lamp or an incandescent lamp. The light source 54 is always turned on when the camera 53 is shooting.

[0048] The hood 55 is a box-shaped member that covers a portion of the upper surface of the receiving conveyor 20. The hood 55 is made of a light-blocking material. The hood 55 is disposed midway along the transport path of the aggregates 9 by the receiving conveyor 20. The hood 55 has an entrance opening 551 and an exit opening 552. The aggregates 9 transported by the receiving conveyor 20 are carried into the hood 55 through the entrance opening 551 and are carried out of the hood 55 through the exit opening 552.

[0049] The camera 53 and the light source 54 are disposed inside a hood 55. The hood 55 blocks external ambient light. This reduces the influence of ambient light when the camera 53 takes an image. Furthermore, by irradiating light from the light source 54 inside the hood 55, the aggregate 9 can be illuminated at a constant brightness. Therefore, even if the amount of sunlight changes depending on the season or weather, the aggregate 9 can be photographed at a constant brightness. Furthermore, it is possible to reduce the change in the shadow of the aggregate 9 due to the position of the sun.

[0050] The information processing unit 56 is a device that performs the learning process and the judgment process described below. The information processing unit 56 is configured by a computer having a processor 561 such as a CPU or a GPU, a memory 562 such as a RAM, and a storage unit 563 such as a hard disk drive or an SSD. A computer program for executing the learning process and the judgment process described below is installed in the storage unit 563. The information processing unit 56 is also connected to the operation panel 51, the display unit 52, the camera 53, the receiving conveyor 20, and the shuttle conveyor 30 described above so as to be able to communicate with each other.

[0051] 2 is a block diagram conceptually showing the functions realized in the information processing unit 56. As shown in FIG. 2, the information processing unit 56 has a learning unit 61, an estimation unit 62, a determination unit 63, and a control unit 64. The functions of these units are realized by the computer serving as the information processing unit 56 operating in accordance with a computer program stored in a storage unit 563.

[0052] The learning unit 61 is a processing unit that generates a learning model M for estimating the type of aggregate based on an image of the aggregate 9. The learning unit 61 performs machine learning using images of sample aggregate 9 prepared for learning (hereinafter referred to as "learning images") and the known aggregate type of the sample aggregate 9 as training data. A convolutional neural network, which is a type of deep learning, is suitable as a machine learning algorithm. In an experiment conducted by the present inventors, when a convolutional neural network having a pooling layer that performs image blurring processing was used as the learning unit 61, a learning model M with sufficient estimation accuracy could be generated even for images that were blurred in the conveying direction.

[0053] However, the machine learning algorithm used by the learning unit 61 is not limited to a convolutional neural network, and may be other supervised machine learning algorithms such as a neural network, linear regression, decision tree, or support vector regression.

[0054] The learning unit 61 learns the relationship between images of a large number of sample aggregates 9 and the aggregate types of the sample aggregates 9. This generates a learning model M that can output an estimation result of the aggregate type corresponding to the image of the aggregate 9. The learning unit 61 provides the generated learning model M to the estimation unit 62. Details of the learning process will be described later.

[0055] The estimation unit 62 is a processing unit that estimates the aggregate type from an image of the aggregate 9 using the above-mentioned learning model M. The estimation unit 62 cuts out an image from the video D captured by the camera 53 and inputs it to the learning model M. The learning model M then outputs an estimation result E of the aggregate type based on the input image of the aggregate 9. The estimation unit 62 sends the estimation result E output from the learning model M to the determination unit 63.

[0056] The determination unit 63 acquires the aggregate type information I input to the operation panel 51. The determination unit 63 also acquires the aggregate type estimation result E by the estimation unit 62. The determination unit 63 then determines whether or not the aggregate type information I input to the operation panel 51 matches the aggregate type estimation result E by the estimation unit 62. The determination unit 63 also outputs the determination result R to the display unit 52. As a result, the display unit 52 displays the determination result R indicating whether or not the aggregate type of the aggregate 9 transported by the receiving conveyor 20 matches the aggregate type specified by the operation panel 51.

[0057] The control unit 64 controls the operation of the receiving conveyor 20 in accordance with the determination result R of the determination unit 63. When the determination unit 63 determines that the aggregate type information I input to the operation panel 51 matches the estimation result E of the estimation unit 62, the control unit 64 continues the operation of the receiving conveyor 20. When the determination unit 63 determines that the aggregate type information I input to the operation panel 51 does not match the estimation result E of the estimation unit 62, the control unit 64 stops the operation of the receiving conveyor 20.

[0058] <1-3. Learning process> Next, a description will be given of the learning process executed in the above-mentioned aggregate determination device 50. Fig. 3 is a flowchart showing the flow of the learning process. This learning process is executed in advance before the determination process of the aggregate 9, which will be described later, is executed.

[0059] When performing the learning process, first, sample aggregates 9 for learning are prepared (step S11). For example, if four aggregate types A to D are to be learned, sample aggregates 9 of the four aggregate types are prepared.

[0060] The operator places the prepared sample aggregates 9 one by one on the receiving conveyor 20 and operates the receiving conveyor 20. Then, while irradiating light from the light source 54, the camera 53 captures a video D of the sample aggregates 9 (step S12). When the operator has completed capturing the video of the sample aggregates 9 of one aggregate type, the operator places the sample aggregates 9 of the next aggregate type on the receiving conveyor 20. Then, the receiving conveyor 20 is operated and the video D of the sample aggregates 9 is captured in the same manner as above. In this way, the videos D of the sample aggregates 9 of all the prepared aggregate types are captured sequentially.

[0061] The video D captured by the camera 53 is made up of a large number of frame images. For example, if the camera 53 has a frame rate of 30 frames / second and captures images for 5 minutes, the video D obtained by the capture will be made up of 9,000 frame images. In step S12, such video D (a collection of frame images) is captured for all prepared aggregate types. The captured video D is then transmitted from the camera 53 to the information processing unit 56.

[0062] Inside the information processing unit 56, a large number of frame images included in the video D are accumulated as learning images in the memory unit 563 (step S13). As described above, the video D includes a large number of frame images. Therefore, the learning unit 61 can prepare a large number of learning images. FIG. 4 is a diagram showing an example of frame images included in the video D. As shown in FIG. 4, due to the relationship between the frame rate of the camera 53 and the moving speed of the belt 23, the frame images are images that have blur in the conveying direction.

[0063] The learning unit 61 classifies the numerous learning images into teacher images and validation images (step S14). For example, if 36,000 learning images are obtained by photographing four types of sample aggregates 9 for five minutes each, three-quarters of the learning images, or 27,000, are used as teacher images, and the remaining one-quarter, or 9,000, are used as validation images. However, the ratio of teacher images to validation images does not necessarily have to be 3:1.

[0064] The learning unit 61 sets a set of a training image and the known aggregate type of the sample aggregate 9 shown in the training image as training data. The learning unit 61 also sets a set of a validation image and the known aggregate type of the sample aggregate 9 shown in the validation image as validation data.

[0065] The learning unit 61 performs image processing such as blurring, left-right flipping, and up-down flipping on the teacher image (step S15). By varying the teacher image through image processing, the range of images to be learned can be expanded. This can improve the accuracy of machine learning. However, it is desirable that the image processing does not include image rotation processing. As mentioned above, the teacher image is an image that has blur in the transport direction, and if rotation processing is performed, the direction of this blur will change, which can actually reduce the accuracy of machine learning. Note that the large number of teacher images may include images that have not been subjected to image processing.

[0066] The learning unit 61 performs machine learning using a large amount of training data after image processing (step S16). That is, the machine learning algorithm inputs a large number of training images one by one into the learning model M. Then, the parameters of the learning model M are adjusted so that the estimated result of the aggregate type output from the learning model M approaches the aggregate type of the training image that was input. The machine learning algorithm repeats this parameter adjustment process for the number of pieces of training data prepared.

[0067] Thereafter, the machine learning algorithm uses a plurality of validation images to confirm the accuracy of the estimation of the aggregate type by the learning model M (step S17). Specifically, the validation images are input to the learning model M, and it is confirmed whether the estimation result of the aggregate type output from the learning model M matches the aggregate type of the input validation images. If the estimation accuracy of the aggregate type reaches a predetermined level (for example, an accuracy rate of 95% or more), the machine learning algorithm ends the learning process. As a result, one learning model M capable of outputting an estimation result of the aggregate type based on the image of the aggregate 9 is generated.

[0068] If the accuracy of the estimation of the aggregate type does not reach a desired level, the processes of steps S12 to S17 described above may be repeated. After generating the learning model M, an image of the sample aggregate 9 other than the training image and validation image described above may be input to the learning model M, and it may be further confirmed whether the estimation result of the aggregate type output from the learning model M matches the aggregate type of the sample aggregate 9 shown in the image.

[0069] Even if the aggregate 9 is the same type of aggregate, the surface color will differ depending on the dry state, wet state, location of crushing, etc. For this reason, in the above-mentioned step S12, video D of sample aggregate 9 in a dry state, video D of sample aggregate 9 in a wet state, and video D of sample aggregate 9 in different locations of crushing may be obtained by photographing multiple times for each type of aggregate.

[0070] Furthermore, the large number of training images input to the learning model M may include images of sample aggregate 9 in a dry state, images of sample aggregate 9 in a wet state, and images of sample aggregate 9 in different crushing locations. For example, when 9,000 training images are used for one aggregate type, the 9,000 images may include a mixture of images of multiple sample aggregates 9 in different dry states, wet states, and crushing locations. This allows the learning unit 61 to learn various states, such as dry states, wet states, and different crushing locations, for each aggregate type. This further improves the estimation accuracy of the aggregate type output from the learning model M.

[0071] <1-4. Aggregate determination process> Next, a process for determining the suitability of the aggregate 9 put into the receiving conveyor 20 using the learning model M when the aggregate 9 is actually received will be described. Fig. 5 is a flowchart showing the flow of the determination process.

[0072] When receiving aggregates 9, first, the aggregates 9 loaded on the truck 90 are dumped into the receiving hopper 10 (step S21). Next, the operator inputs information I about the aggregate type of the dumped aggregates 9 into the operation panel 51 (step S22). For example, the operator inputs the aggregate type information I by pressing key 511 on the operation panel 51. Thereafter, the operator presses key 512 on the operation panel 51 to instruct the start of conveyance. This starts the operation of the receiving conveyor 20 (step S23).

[0073] The aggregates 9 supplied from the receiving hopper 10 to the upper surface of the belt 23 are transported obliquely upward toward the terminal pulley 22 as the belt 23 rotates. Then, after a predetermined time has elapsed since the operation of the receiving conveyor 20 started, the camera 53 captures a video D of the aggregates 9 as the aggregates 9 pass below the camera 53 (step S24). The camera 53 then transmits the obtained video D to the information processing unit 56.

[0074] Within the information processing unit 56, the estimation unit 62 cuts out one image (frame image) from the video D (step S25). Then, the estimation unit 62 inputs the cut-out image to the learning model M. Then, the learning model M outputs an estimation result E of the aggregate type based on the input image of the aggregate 9 (step S26). The estimation unit 62 sends the estimation result E output from the learning model M to the determination unit 63.

[0075] The determination unit 63 acquires the aggregate type information I input to the operation panel 51 in the above step S22. The determination unit 63 also acquires the estimation result E output from the learning model M in the above step S26. Then, the determination unit 63 determines whether or not the aggregate type information I input to the operation panel 51 matches the aggregate type estimation result E by the estimation unit 62 (step S27).

[0076] The determination unit 63 outputs the determination result R to the display unit 52. As a result, the display unit 52 displays the determination result R indicating whether or not the aggregate type of the aggregate 9 transported by the receiving conveyor 20 matches the aggregate type specified by the operation panel 51 (step S28). The operator can recognize the determination result R by checking the display unit 52.

[0077] The control unit 64 checks whether the determination result R of the determination unit 63 in step S27 above is "match" or "mismatch" (step S29). If the determination result R indicates that the aggregate type information I input to the operation panel 51 matches the estimation result E of the estimation unit 62 (step S29: Yes), the control unit 64 continues the operation of the receiving conveyor 20. On the other hand, if the determination result R indicates that the aggregate type information I input to the operation panel 51 does not match the estimation result E of the estimation unit 62 (step S29: No), the control unit 64 stops the operation of the receiving conveyor 20 (step S30).

[0078] If the judgment result R is "mismatch," it is possible that aggregate 9 of an inappropriate aggregate type has been put into the receiving hopper 10, or that the aggregate type information I has been incorrectly entered into the operation panel 51. In either case, there is a possibility that the aggregate 9 on the receiving hopper 10 will be transported to the wrong silo 40. In such a case, in step S30 above, the operation of the receiving conveyor 20 is brought to an emergency stop. This stops the transport of the aggregate 9 and prevents the aggregate 9 from being transported to the wrong silo 40.

[0079] As described above, the aggregate determination device 50 photographs the aggregate 9 being transported by the receiving conveyor 20, and estimates the aggregate type of the aggregate 9 being transported using the obtained image and the learning model M previously generated by machine learning. The aggregate determination device 50 then determines whether the aggregate type estimation result E matches the aggregate type information I input to the operation panel 51. This makes it possible to detect the possibility that the aggregate 9 will be transported to the wrong silo 40.

[0080] In particular, in the aggregate identification device 50 of this embodiment, the aggregate 9 is photographed without stopping the receiving conveyor 20. As a result, the image input to the learning model M is not a high-resolution image but an image with blur. For the camera 53 that photographs the aggregate 9, an easily available, inexpensive camera can be used instead of a high-speed camera with a high frame rate that can acquire frame images without blur in the conveyance direction or a high-performance camera with a large number of pixels. This reduces the cost of the camera 53. Furthermore, by using an easily available, inexpensive camera 53, the cost of replacing the camera 53 when it breaks down can also be reduced. Therefore, an aggregate identification device 50 with excellent maintainability can be realized.

[0081] Furthermore, in the aggregate discrimination device 50 of this embodiment, a convolutional neural network that performs image blurring processing is used to generate the learning model M in the learning unit 61. Therefore, the image input to the learning model M does not necessarily need to be a high-resolution image. That is, even if an image is blurred in the conveyance direction, a learning model M with high estimation accuracy can be obtained. Therefore, a learning model M with high estimation accuracy can be obtained using an inexpensive camera 53 instead of a high-performance camera. The camera 53 may be, for example, a webcam with a frame rate of approximately 30 frames per second. Furthermore, by reducing the frame rate and the number of pixels, the memory capacity of the aggregate discrimination device 50 can be reduced. That is, an aggregate discrimination device with sufficient discrimination accuracy can be realized despite its inexpensive configuration.

[0082] If the frame rate of the camera 53 is 30 frames per second, frame images with blur in the conveyance direction can be acquired by setting the moving speed of the belt 23 of the receiving conveyor 20 to 50 m / min or more (moving distance per frame: 2.78 cm / frame). Furthermore, if the moving speed of the belt 23 exceeds 120 m / min (moving distance per frame: 6.67 cm / frame), the frame images obtained by shooting will be blurred to the extent that the outline of the aggregate cannot be discerned. Therefore, it is recommended to adjust the relationship between the frame rate of the camera 53 and the moving speed of the belt 23 so that the moving distance per frame is in the range of 2.78 cm / frame or more and 6.67 cm / frame or less.

[0083] Furthermore, in the aggregate determination device 50 of this embodiment, an image is captured by the camera 53 while irradiating light from the light source 54 inside the hood 55. This makes it possible to reduce variations in brightness of the image input to the learning model M. Therefore, the type of aggregate can be estimated more accurately.

[0084] 2. Second embodiment (moisture sensor) Next, a second embodiment of the present invention will be described. Fig. 6 is a diagram showing the configuration of an aggregate transporting device 1 according to the second embodiment.

[0085] This embodiment differs from the first embodiment in that the aggregate determination device 50 includes a moisture sensor 57. The moisture sensor 57 is a sensor that measures the amount of moisture on the surface of the aggregate 9 transported by the receiving conveyor 20. The moisture sensor 57 is disposed above the receiving conveyor 20, upstream of the camera 53. The moisture sensor 57 is also disposed within the hood 55 together with the camera 53 and the light source 54.

[0086] For example, a near-infrared moisture meter is used as the moisture sensor 57. The near-infrared moisture meter measures the moisture content of the aggregate 9 by irradiating near-infrared rays onto the aggregate 9 and measuring the absorbance at the surface of the aggregate 9 based on the amount of reflected light. However, the moisture sensor 57 may be a moisture meter of another type. The moisture sensor 57 is connected to the information processing unit 56 so as to be able to communicate with the information processing unit 56. The measurement result of the moisture sensor 57 is transmitted to the information processing unit 56.

[0087] Fig. 7 is a flowchart showing the flow of the learning process in the second embodiment. As shown in Fig. 7, in this second embodiment, after preparing a sample aggregate 9, first, the same processes as steps S12 to S17 described above are performed on the dry sample aggregate 9 (steps S12A to S17A). As a result, a first learning model M1 capable of outputting an aggregate-type estimation result E based on an image of the dry aggregate 9 is generated. Next, the same processes as steps S12 to S17 described above are performed on the wet sample aggregate 9 (steps S12B to S17B). As a result, a second learning model M2 capable of outputting an aggregate-type estimation result E based on an image of the wet aggregate 9 is generated.

[0088] The "dry state" refers to a state in which the amount of moisture on the surface of the aggregate 9 measured by the moisture sensor 57 is less than a predetermined value. The "wet state" refers to a state in which the amount of moisture on the surface of the aggregate 9 measured by the moisture sensor 57 is equal to or greater than a predetermined value.

[0089] Fig. 8 is a flowchart showing the flow of the determination process in the second embodiment. As shown in Fig. 8, in the second embodiment, similarly to the first embodiment, the processes of steps S21 to S25 are performed to acquire an image of the aggregate 9. However, immediately before the image of the aggregate 9 is captured by the camera 53, the moisture sensor 57 measures the moisture content on the surface of the aggregate 9 (step S24A).

[0090] Thereafter, the estimation unit 62 determines whether the aggregate 9 transported by the receiving conveyor 20 is in a dry state or a wet state (step S31). If the moisture content measured by the moisture sensor 57 is less than the predetermined value described above, the estimation unit 62 determines that the aggregate 9 is in a dry state (step S31: Yes). If the moisture content measured by the moisture sensor 57 is equal to or greater than the predetermined value described above, the estimation unit 62 determines that the aggregate 9 is in a wet state (step S31: No).

[0091] The estimation unit 62 selects the first learning model M1 when the aggregate 9 is dry (step S32), and selects the second learning model M2 when the aggregate 9 is wet (step S33).

[0092] When the first learning model M1 is selected, the estimation unit 62 inputs an image of the aggregate 9 to the first learning model M1. The first learning model M1 then outputs an estimation result E of the aggregate type based on the input image (step S26). When the second learning model M2 is selected, the estimation unit 62 inputs an image of the aggregate 9 to the second learning model M2. The second learning model M2 then outputs an estimation result E of the aggregate type based on the input image (step S26).

[0093] Thereafter, the aggregate determination device 50 performs the processes of steps S27 to S30 in the same manner as in the first embodiment.

[0094] As described above, in the second embodiment, the aggregate determination device 50 includes a moisture sensor 57. When the aggregate 9 is dry, the first learning model M1, whose parameters are adjusted for the dry state, is used. When the aggregate 9 is wet, the second learning model M2, whose parameters are adjusted for the wet state, is used. In this way, by using different learning models M1 and M2 for the dry and wet states, the aggregate type can be estimated with higher accuracy.

[0095] <3. Third embodiment (majority vote)> Next, a third embodiment of the present invention will be described. Fig. 9 is a flowchart showing the flow of the determination process in the third embodiment. As shown in Fig. 9, in this third embodiment, similarly to the first embodiment, the processes of steps S21 to S24 are performed to acquire a moving image D of the aggregate 9.

[0096] The estimation unit 62 cuts out an odd number of images (an odd number equal to or greater than three) from the video D (step S25). Then, the estimation unit 62 inputs each of the cut out odd number of images to the learning model M. The learning model M then outputs an odd number of estimation results E corresponding to each of the input odd number of images of aggregate 9 (step S26). The estimation unit 62 sends the odd number of estimation results E output from the learning model M to the determination unit 63.

[0097] Next, the determination unit 63 compares whether or not each of the odd number of estimation results E matches the aggregate type information I input to the operation panel 51. Then, the determination unit 63 makes a final determination by majority vote of the comparison results (step S27). That is, if there are more "matches" than "mismatches" among the odd number of comparison results, the determination unit 63 determines the final determination result R to be "matches." On the other hand, if there are more "mismatches" than "matches" among the odd number of comparison results, the determination unit 63 determines the final determination result R to be "mismatches."

[0098] Thereafter, the aggregate determination device 50 performs the processes of steps S28 to S30 in the same manner as in the first embodiment.

[0099] As described above, in the third embodiment, an odd number of estimation results E are output from the learning model M based on an odd number of images of aggregates 9. Then, based on the odd number of estimation results E, it is determined by majority vote whether the aggregate 9 fed onto the receiving conveyor 20 matches the aggregate type input to the operation panel 51. This makes it possible to obtain a more accurate determination result R.

[0100] The aggregate determination device 50 may have an odd number of cameras 53, three or more. An odd number of images may be acquired by cutting out one image from each of the videos D captured by the odd number of cameras 53.

[0101] <4. Fourth embodiment (additional learning)> Next, a fourth embodiment of the present invention will be described. This fourth embodiment differs from the first embodiment in that additional machine learning is performed as appropriate while performing the above-mentioned determination process using a learning model M that has been generated. Fig. 10 is a flowchart showing the flow of processing when additional machine learning is performed.

[0102] 10, after the judgment result R is displayed on the display unit 52, the operator determines whether or not the displayed judgment result R is correct. Then, the operator inputs correct / incorrect information indicating whether or not the judgment result R is correct into the operation panel 51 (step S34).

[0103] For example, when the judgment result R of "mismatch" is displayed on the display unit 52, the operator checks whether or not there is indeed a mismatch between the aggregate type of the aggregate 9 fed onto the receiving conveyor 20 and the aggregate type information I input into the operation panel 51. If there is indeed a mismatch, the operator inputs into the operation panel 51 that the judgment result R by the judgment unit 63 is correct. On the other hand, when the aggregate type of the aggregate 9 fed onto the receiving conveyor 20 matches the aggregate type information I input into the operation panel 51, the operator inputs into the operation panel 51 that the judgment result R by the judgment unit 63 is incorrect.

[0104] The correct / incorrect information input to the operation panel 51 is transmitted to the information processing unit 56. The information processing unit 56 stores a set of the image cut out in step S25, the estimation result in step S26, and the correct / incorrect information input in step S34 in the memory unit 563 as training data for additional learning (step S35). The information processing unit 56 stores such training data every time the receiving conveyor 20 receives aggregate 9.

[0105] In step S35, not only the images (frame images) extracted in step S25 but also a video D of a predetermined time (e.g., 5 minutes) including the images may be included in the training data for additional learning. In this way, many frame images included in the video D can be used as images for additional learning. Therefore, a large amount of training data can be accumulated in a short period of time.

[0106] Thereafter, the learning unit 61 of the information processing unit 56 performs additional machine learning using the accumulated training data at a predetermined timing (step S36). The timing for performing additional machine learning may be, for example, when the rate at which the above-mentioned true / false information is "incorrect" becomes higher than a predetermined threshold. The learning unit 61 updates or recreates the learning model M using only the training data for additional learning, or the initial training data plus the training data for additional learning. This allows the estimation accuracy of the aggregate type by the learning model M to be maintained at a predetermined level.

[0107] Furthermore, when the moisture amount on the surface of the aggregate 9 is measured by the moisture sensor 57 as in the second embodiment, the measured moisture amount may be included in the training data for additional learning.

[0108] <5. Variations> Although the first to fourth embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments.

[0109] In the above embodiment, the learning unit 61 performs image processing on the frame images, such as blurring, left-right inversion, and up-down inversion. However, the frame images may be used as learning images as they are without performing any image processing.

[0110] Furthermore, in the above embodiment, the camera 53 captures the video D of the aggregates 9 being transported by the receiving conveyor 20. However, the camera 53 may also capture a still image of the aggregates 9 being transported by the receiving conveyor 20. Even when capturing a still image instead of the video D, it is possible to obtain an image that includes blur by capturing the image without stopping the transport of the aggregates 9. In other words, the camera 53 is not limited to capturing the video D, and may also capture a still image of the aggregates 9 being transported to obtain an image that includes blur as a learning image.

[0111] In the above embodiment, the receiving conveyor 20 is stopped when the determination result R by the determination unit 63 is "mismatch." However, for example, a cut gate may be provided below the receiving hopper 10, and the cut gate may be closed when the determination result R is "mismatch." A surge bin for temporarily storing the aggregate 9 may be provided on the conveyance path, and the aggregate 9 may be conveyed to the surge bin when the determination result R is "mismatch." A gate may be provided somewhere on the conveyance path to the silo 40, and the gate may be closed when the determination result R is "mismatch." That is, the control unit 64 may restrict the conveyance of the aggregate 9 to the silo 40 in some way when the determination result R is "mismatch."

[0112] In the above embodiment, the receiving conveyor 20 is inclined. However, the receiving conveyor 20 does not have to be inclined. In other words, the receiving conveyor 20 may transport the aggregates 9 in a horizontal direction.

[0113] In the above embodiment, the computer serving as the information processing unit 56 is located within the concrete plant. However, some or all of the functions of the information processing unit 56 may be located outside the concrete plant. For example, the computer serving as the information processing unit 56 may be located in a remote location and may be able to communicate with the operation panel 51, the display unit 52, and the camera 53 via the Internet.

[0114] Furthermore, in the above embodiment, the judgment result R of the judgment unit 63 is displayed on the display unit 52. However, when the judgment result R of "mismatch" is output from the judgment unit 63, a warning lamp may be turned on or a warning buzzer may be sounded. In other words, the "result output unit" in the present invention may be a warning lamp, a warning buzzer, or the like.

[0115] In the above embodiment, an example has been described in which the type of aggregate is determined on the receiving conveyor 20 of a concrete plant. However, the present invention can also be used in a quarry that ships and supplies aggregate 9 to a concrete plant. In other words, the aggregate determination device of the present invention may determine the type of aggregate on a conveyor installed in a quarry.

[0116] Furthermore, the elements appearing in the above-described embodiments and modifications may be combined as appropriate within the scope of not causing any contradiction. [Industrial Applicability]

[0117] The present invention can be used to determine aggregate type in a concrete plant or quarry. [Explanation of symbols]

[0118] 1 Aggregate conveying device 9 Aggregate 10 Receiving hopper 20 Receiving conveyor 30 Shuttle Conveyor 40 Silo 50 Aggregate determination device 51 Control panel 52 Display section 53 Camera 54 Light source 55 Food 56 Information Processing Department 57 Moisture Sensor 61 Learning Department 62 Estimation part 63 Judgment section 64 Control Unit 90 Tracks D Video E Estimation results I. Aggregate type information M Learning Model M1 First learning model M2 Second learning model R judgment result

Claims

1. An aggregate judgment device that judges whether aggregate fed into a conveyor is suitable or not, a camera for photographing the aggregate transported by the conveyor; an operation panel into which information on aggregate types is input; an information processing unit communicably connected to the camera and the operation panel; Equipped with The information processing unit an estimation unit that inputs images of aggregates captured by the camera into a learning model generated by machine learning and outputs an estimation result of the aggregate type from the learning model; a determination unit that determines whether the aggregate type input to the operation panel matches the estimated result; and the camera photographs the aggregate being transported by the conveyor to obtain an image blurred in the transport direction of the aggregate; the learning model is generated by performing machine learning using, as training data, images obtained by photographing the sample aggregate with the camera without rotating the images; The camera captures video of the aggregate being transported by the conveyor; the video includes a plurality of frame images having a blur in the conveyance direction of the aggregate, the information processing unit inputs the frame images included in the video captured by the camera into the learning model; The information processing unit a learning unit that generates the learning model by performing machine learning using a plurality of frame images included in a video of a sample aggregate prepared for learning and known aggregate types of the sample aggregate as training data; and The learning unit performs the machine learning after performing image processing other than rotation on the frame images.

2. The aggregate determination device according to claim 1, the operation panel is capable of inputting correct / incorrect information indicating whether the determination result by the determination unit is correct or not; The learning unit updates the learning model by performing additional machine learning using the images of aggregate obtained from the camera, the estimation results, and the accuracy information as training data.

3. The aggregate determination device according to claim 1 or 2, A light source that irradiates light toward the camera's shooting range The aggregate determination device further comprises:

4. The aggregate determination device according to claim 3, A hood that covers a portion of the top surface of the conveyor Furthermore, The aggregate determination device, wherein the camera and the light source are disposed within the hood.

5. The aggregate determination device according to any one of claims 1 to 4, a result output unit that outputs the determination result of the determination unit The aggregate determination device further comprises:

6. An aggregate judgment device that judges whether aggregate fed into a conveyor is suitable or not, a camera for photographing the aggregate transported by the conveyor; an operation panel into which information on aggregate types is input; an information processing unit communicably connected to the camera and the operation panel; Equipped with The information processing unit an estimation unit that inputs images of aggregates captured by the camera into a learning model generated by machine learning and outputs an estimation result of the aggregate type from the learning model; a determination unit that determines whether the aggregate type input to the operation panel matches the estimated result; and the camera photographs the aggregate being transported by the conveyor to obtain an image blurred in the transport direction of the aggregate; the learning model is generated by performing machine learning using, as training data, images obtained by photographing the sample aggregate with the camera without rotating the images; A moisture sensor that measures the moisture content on the surface of the aggregate transported by the conveyor. Furthermore, The information processing unit a first learning model generated by performing the machine learning on aggregates having a surface moisture content less than a predetermined value; a second learning model generated by performing the machine learning on aggregates having a surface moisture content equal to or greater than the predetermined value; and The estimation unit If the moisture content measured by the moisture sensor is less than the predetermined value, an image of the aggregate taken by the camera is input to the first learning model, and an estimation result of the aggregate type is output from the first learning model; An aggregate determination device that, when the moisture content measured by the moisture sensor is greater than or equal to the predetermined value, inputs the image of the aggregate obtained from the camera into the second learning model and outputs an estimation result of the aggregate type from the second learning model.

7. The aggregate determination device according to any one of claims 1 to 6, the estimation unit inputs an odd number of images obtained from the camera to the learning model, and outputs an odd number of estimation results corresponding to each of the odd number of images from the learning model; The determination unit compares the aggregate type input to the operation panel with the odd number of estimated results and makes a determination by majority vote of the comparison results.

8. The aggregate determination device according to any one of claims 1 to 7, The information processing unit a control unit that limits the transport of aggregate to a silo when the determination unit determines that the aggregate type input to the operation panel does not match the estimated result; The aggregate determination device further comprises:

9. An aggregate judgment method for judging the suitability of aggregate fed into a conveyor, comprising: a) inputting information on the type of aggregate of the aggregate fed into the conveyor into an operation panel; b) taking an image of the aggregate being transported by the conveyor; c) inputting the images of the aggregate obtained by photographing in step b) into a learning model generated by machine learning, and outputting an estimation result of the aggregate type from the learning model; d) determining whether the aggregate type input in step a) matches the estimated result; and In the step b), a camera photographs the aggregate being transported by the conveyor to obtain an image blurred in the transport direction of the aggregate; the learning model is generated by performing machine learning using, as training data, images obtained by photographing the sample aggregate with the camera without rotating the images; The camera captures video of the aggregate being transported by the conveyor; the video includes a plurality of frame images having a blur in the conveyance direction of the aggregate, In the step c), the frame images included in the video captured by the camera are input to the learning model; e) A learning process of generating the learning model by performing machine learning using a plurality of frame images included in a video of a sample aggregate prepared for learning and the known aggregate type of the sample aggregate as training data. and In the learning step, the frame images are subjected to image processing other than rotation, and then the machine learning is performed.

10. An aggregate judgment method for judging the suitability of aggregate fed into a conveyor, comprising: a) inputting information on the type of aggregate of the aggregate fed into the conveyor into an operation panel; b) taking an image of the aggregate being transported by the conveyor; c) inputting the images of the aggregate obtained by photographing in step b) into a learning model generated by machine learning, and outputting an estimation result of the aggregate type from the learning model; d) determining whether the aggregate type input in step a) matches the estimated result; and In the step b), a camera photographs the aggregate being transported by the conveyor to obtain an image blurred in the transport direction of the aggregate; the learning model is generated by performing machine learning using, as training data, images obtained by photographing the sample aggregate with the camera without rotating the images; In the step b), the moisture content of the surface of the aggregate transported by the conveyor is measured by a moisture sensor; The learning model is a first learning model generated by performing the machine learning on aggregates having a surface moisture content less than a predetermined value; a second learning model generated by performing the machine learning on aggregates having a surface moisture content equal to or greater than the predetermined value; and In the step c), If the moisture content measured by the moisture sensor is less than the predetermined value, an image of the aggregate taken by the camera is input to the first learning model, and an estimation result of the aggregate type is output from the first learning model; An aggregate determination method in which, when the moisture content measured by the moisture sensor is greater than or equal to the predetermined value, an image of the aggregate obtained from the camera is input into the second learning model, and an estimation result of the aggregate type is output from the second learning model.

Images