Aggregate determination device and aggregate determination method

Abstract

To provide an aggregate determination device and an aggregate determination method that can determine whether an aggregate introduced to a conveyor matches aggregate types input to a control panel.SOLUTION: An aggregate determination device 50 comprises a control panel 51, a camera 53, and an information processing unit 56. The camera 53 photographs an aggregate 9 conveyed by a reception conveyor 20. An operator inputs, to the control panel 51, information on the aggregate type of the aggregate 9 introduced to the reception conveyor 20. The information processing unit 56 inputs an image of the aggregate 9 obtained through the photographing performed by the camera 53 to a learning model generated through machine learning. Consequently, a result of estimation of the aggregate type is output from the learning model. The information processing unit 56 determines whether the information on the aggregate type input to the control panel 51 and the estimation result output from the learning model match each other.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an aggregate determination device and an aggregate determination method for determining the suitability of aggregates input to a conveyor.

Background Art

[0002] Conventionally, a concrete plant is provided with a plurality of silos for storing aggregates that are raw materials for concrete. Aggregates transported by truck from a quarry are conveyed by a conveyor in the concrete plant and stored in designated silos for each aggregate type. A conventional concrete plant is described in, for example, Patent Document 1.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Aggregates are classified into a plurality of aggregate types according to characteristics such as particle size, color, and place of origin. In a concrete plant, a silo to be stored is determined for each aggregate type. When aggregates are input from a truck to a conveyor, the driver of the truck or a predetermined worker inputs the aggregate type to an operation panel. Thereby, the conveyance destination of the aggregates by the conveyor is set to a silo corresponding to the aggregate type.

[0005]

[0006] The present invention has been made in view of such circumstances, and an object thereof is to provide an aggregate determination device and an aggregate determination method capable of determining whether or not the aggregate input to the conveyor matches the aggregate type input to the operation panel.

Means for Solving the Problems

[0007] To solve the above problems, a first invention of the present application is an aggregate determination device for determining the suitability of an aggregate input to a conveyor, including a camera that photographs the aggregate conveyed by the conveyor, an operation panel into which information on the aggregate type is input, and an information processing unit communicably connected to the camera and the operation panel. The information processing unit inputs an image of the aggregate obtained by photographing with the camera into a learning model generated by machine learning, and outputs an estimation result of the aggregate type from the learning model. The information processing unit also has a determination unit that determines whether or not the aggregate type input to the operation panel matches the estimation result.

[0008] A second invention of the present application is the aggregate determination device of the first invention, wherein the camera photographs a moving image of the aggregate conveyed by the conveyor, the moving image includes a plurality of frame images having blurring in the conveying direction of the aggregate, and the information processing unit inputs the frame images included in the moving image photographed by the camera into the learning model.

[0009] A third invention of the present application is the aggregate determination device of the second invention, wherein the information processing unit further has a learning unit that generates the learning model by performing machine learning using, as teacher data, a plurality of frame images included in a moving image of a sample aggregate prepared for learning and the known aggregate type of the sample aggregate.

[0010] A fourth invention of the present application is the aggregate determination device of the third invention, wherein the learning unit performs the machine learning after performing image processing other than rotation on the frame image.

[0011] The fifth invention of the present application is the aggregate determination device of the third or fourth invention, wherein the operation panel can input correct / incorrect information indicating whether the determination result by the determination unit is correct, and the learning unit uses the image of the aggregate obtained from the camera, the estimation result, and the correct / incorrect information as teacher data to perform additional machine learning to update the learning model.

[0012] The sixth invention of the present application is the aggregate determination device of any one of the first to fifth inventions, further comprising a light source that irradiates light toward the shooting range of the camera.

[0013] The seventh invention of the present application is the aggregate determination device of the sixth invention, further comprising a hood that covers a part of the upper surface of the conveyor, and the camera and the light source are arranged inside the hood.

[0014] The eighth invention of the present application is the aggregate determination device of any one of the first to seventh inventions, further having a result output unit that outputs the determination result of the determination unit.

[0015] The ninth invention of the present application is the aggregate determination device of any one of the first to eighth inventions, further comprising a moisture sensor that measures the moisture content on the surface of the aggregate conveyed by the conveyor, and the information processing unit has a first learning model generated by performing the machine learning on aggregates with a surface moisture content less than a predetermined value and a second learning model generated by performing the machine learning on aggregates with a surface moisture content greater than or equal to the predetermined value. When the moisture content measured by the moisture sensor is less than the predetermined value, the estimation unit inputs the image of the aggregate captured by the camera into the first learning model and outputs an estimation result of the aggregate type from the first learning model. When the moisture content measured by the moisture sensor is greater than or equal to the predetermined value, the estimation unit inputs the image of the aggregate obtained from the camera into the second learning model and outputs an estimation result of the aggregate type from the second learning model.

[0016] The 10th invention of the present application is an aggregate determination device according to any one of the 1st to 9th inventions, wherein the estimation unit inputs an odd number of images obtained from the camera into the learning model, and outputs an odd number of estimation results corresponding to each of the odd number of images from the learning model. The determination unit compares the aggregate type input to the operation panel with the odd number of estimation results, and makes a determination based on a majority vote of the comparison results.

[0017] The 11th invention of the present application is an aggregate determination device according to any one of the 1st to 10th inventions, wherein the information processing unit further includes a control unit that restricts the conveyance of the aggregate to the silo when the determination unit determines that the aggregate type input to the operation panel does not match the estimation result.

[0018] The 12th invention of the present application is an aggregate determination method for determining the suitability of an aggregate input to a conveyor, including: a) a step of inputting information on the aggregate type of the aggregate input to the conveyor into an operation panel; b) a step of photographing the aggregate conveyed by the conveyor; c) a step of inputting the image of the aggregate obtained by the photographing in step b) into a learning model generated by machine learning, and outputting an estimation result of the aggregate type from the learning model; and d) a step of determining whether or not the aggregate type input in step a) matches the estimation result.

Advantages of the Invention

[0019] According to the 1st to 12th inventions of the present application, it is possible to determine whether the aggregate input to the conveyor matches the aggregate type input to the operation panel.

[0020] In particular, according to the second invention of the present application, a video of the aggregate is taken, and a frame image having blur in the conveying direction is input into the learning model. The camera for photographing the aggregate does not need to be a high-speed camera with a high frame rate capable of acquiring a frame image without blur in the conveying direction or a high-performance camera with a large number of pixels. Instead, an inexpensive and easily available camera is sufficient. This can reduce the cost of the camera. Also, by using an inexpensive and easily available camera, the replacement cost when the camera fails can be suppressed. Therefore, an aggregate determination device with excellent maintainability can be realized.

[0021] In particular, according to the third invention of the present application, a large number of frame images included in the video are used as learning images. For this reason, a large number of aggregate images necessary for generating the learning model can be acquired in a relatively short time. Also, a learning model having sufficient estimation accuracy can be created with frame images having blur. Further, the storage capacity of the aggregate determination device can be reduced compared to the case of preparing a large number of fine images. That is, a learning model with high estimation accuracy can be generated with an inexpensive device configuration.

[0022] In particular, according to the fifth invention of the present application, the learning model can be updated by performing additional machine learning. Thereby, the estimation accuracy of the aggregate type by the learning model can be maintained at a certain level. Also, by accumulating aggregate images, estimation results, and correct / incorrect information, additional machine learning can be performed quickly.

[0023] In particular, according to the sixth invention of the present application, by suppressing the variation in brightness, the aggregate type can be estimated more accurately.

[0024] In particular, according to the seventh invention of the present application, by suppressing the influence of external ambient light, the aggregate type can be estimated more accurately.

[0025] In particular, according to the eighth invention of the present application, the determination result of the determination unit can be recognized by the user.

[0026] In particular, according to the ninth invention of the present application, different learning models are used depending on whether the moisture content on the surface of the aggregate is less than a predetermined value or not less than the predetermined value. Thereby, the aggregate type can be estimated more accurately.

[0027] In particular, according to the tenth invention of the present application, a more accurate estimation result can be obtained based on an odd number of images.

[0028] In particular, according to the eleventh invention of the present application, when there is a possibility that inappropriate aggregates are input, the conveyance of the aggregates to the silo can be restricted.

Brief Description of the Drawings

[0029]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Embodiments for Carrying Out the Invention

[0030] Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings.

[0031] <1. First Embodiment> <1-1. Configuration of Aggregate Conveyor> FIG. 1 is a diagram showing the configuration of an aggregate conveyor 1 including an aggregate determination device 50. This aggregate conveyor 1 is a device that receives aggregate 9 as a raw material in a concrete plant and conveys the aggregate 9 to a plurality of silos 40. As shown in FIG. 1, the aggregate conveyor 1 includes a receiving hopper 10, a receiving conveyor 20, a shuttle conveyor 30, four silos 40, and an aggregate determination device 50.

[0032] The receiving hopper 10 is disposed above the conveying start position of the receiving conveyor 20. The receiving hopper 10 is a funnel-shaped hopper whose inner diameter gradually decreases downward. The aggregate 9 is transported from a quarry by a truck 90 and is introduced into the upper opening of the receiving hopper 10. The introduced aggregate 9 is temporarily stored in the receiving hopper 10 and is supplied from the lower opening of the receiving hopper 10 to the upper surface of the receiving conveyor 20.

[0033] The receiving conveyor 20 is a mechanism that conveys the aggregate 9 introduced into the receiving hopper 10 obliquely upward. The receiving conveyor 20 has a starting pulley 21, an ending pulley 22, and an annular belt 23 stretched between these pulleys 21, 22. The starting pulley 21 is located below the receiving hopper 10. The ending pulley 22 is located above the shuttle conveyor 30 described later. The ending pulley 22 is disposed at a position obliquely upward and away from the starting pulley 21.

[0034] At least one of the starting pulley 21 and the ending pulley 22 rotates by the power of a motor (not shown). Thereby, the belt 23 rotates between the starting pulley 21 and the ending pulley 22 in the direction of the arrow in FIG. 1. The moving speed of the belt 23 is adjusted to an arbitrary speed of, for example, 50 m / min or more. The aggregate 9 fed into the receiving hopper 10 is supplied to the upper surface of the belt 23 near the starting pulley 21. Then, the aggregate 9 placed on the belt 23 moves obliquely upward from the position below the receiving hopper 10 toward the ending pulley 22 as the belt 23 rotates.

[0035] Note that the belt 23 of the receiving conveyor 20 moves continuously at a constant speed. For this reason, the aggregate 9 drawn out from the lower part of the receiving hopper 10 is conveyed while being placed on the upper surface of the belt 23 at a substantially constant height. When the aggregate 9 is conveyed to the position of the ending pulley 22, the aggregate 9 falls onto the shuttle conveyor 30 as the belt 23 reverses.

[0036] The shuttle conveyor 30 is a mechanism for distributing the aggregate 9 conveyed from the receiving conveyor 20 to the four silos 40. The shuttle conveyor 30 is located below the ending pulley 22 of the receiving conveyor 20. The shuttle conveyor 30 has a first pulley 31, a second pulley 32, and an annular belt 33 stretched between these pulleys 31, 32.

[0037] At least one of the first pulley 31 and the second pulley 32 rotates by the power of a motor (not shown). Thereby, the belt 33 rotates between the first pulley 31 and the second pulley 32. The aggregate 9 that has fallen from the receiving conveyor 20 and is placed on the belt 33 of the shuttle conveyor 30 is conveyed to any one of the four silos 40 by the rotation of the belt 33.

[0038] Also, the shuttle conveyor 30 is movable along a horizontal plane by a drive mechanism (not shown). Thereby, the conveyance destination of the aggregate 9 by the shuttle conveyor 30 can be switched to a designated silo 40 among the four silos 40.

[0039] The four silos 40 are storage tanks for the aggregates 9. The aggregates 9 are classified into a plurality of aggregate types according to characteristics such as particle size, color, and place of origin. In this aggregate conveying device 1, a silo 40 to be stored is determined for each aggregate type. In the example of FIG. 1, the aggregates 9 of four aggregate types A to D are separately stored in the four silos 40. Note that the number of silos 40 provided in the aggregate conveying device 1 is not limited to four.

[0040] <Regarding the aggregate determination device> The aggregate determination device 50 is a device for determining the suitability of the aggregates 9 fed into the receiving conveyor 20. The aggregate determination device 50 photographs the aggregates 9 conveyed by the receiving conveyor 20 and estimates the aggregate type based on the obtained image. Then, the aggregate determination device 50 determines whether the estimated result of the aggregate type matches the designated aggregate type. As shown in FIG. 1, the aggregate determination device 50 includes an operation panel 51, a display unit 52, a camera 53, a light source 54, a hood 55, and an information processing unit 56.

[0041] The operation panel 51 is arranged near the receiving hopper 10. The operation panel 51 is electrically connected to the information processing unit 56. The operation panel 51 has a plurality of keys 511, 512. The operator (the driver of the truck 90 or the worker of the concrete plant) presses the key 511 indicating the aggregate type of the aggregates 9 fed into the receiving hopper 10. Thereby, the information of the aggregate type is input from the operation panel 51 to the information processing unit 56. Further, when the information of the aggregate type is input, the shuttle conveyor 30 changes its position so that the silo 40 of the said aggregate type becomes the conveyance destination.

[0042] Also, after the aggregates 9 are fed into the receiving hopper 10, the operator presses the key 512 for instructing the start of conveyance. Then, the operations of the receiving conveyor 20 and the shuttle conveyor 30 are started. Thereby, the fed aggregates 9 are conveyed to the designated silo 40 by the receiving conveyor 20 and the shuttle conveyor 30.

[0043] Note that the operation panel 51 may be configured by a general keyboard, mouse, or touch panel.

[0044] The display unit 52 is arranged near the receiving hopper 10 together with the operation panel 51. The display unit 52 is electrically connected to the information processing unit 56. For example, a liquid crystal display is used for the display unit 52. The display unit 52 displays various information regarding the aggregate determination device 50. For example, the display unit 52 displays the determination result output from the determination unit 63 (to be described later) of the information processing unit 56. That is, the display unit 52 is an example of the "result output unit" in the present invention.

[0045] The camera 53 is an imaging device that photographs the aggregate 9 conveyed by the receiving conveyor 20. The camera 53 is arranged so as to face the upper surface of the belt 23 of the receiving conveyor 20 on the downstream side of the conveying path from the receiving hopper 10. The camera 53 of the present embodiment photographs a moving image of the aggregate 9 conveyed by the receiving conveyor 20. After the operation of the receiving conveyor 20 is started and a predetermined time has elapsed and the leading aggregate 9 passes below the camera 53, the camera 53 starts photographing the moving image. Then, the camera 53 inputs the obtained moving image to the information processing unit 56.

[0046] Note that a sensor for detecting the aggregate 9 may be provided in the middle of the receiving conveyor 20, and the photographing of the moving image by the camera 53 may be started based on the detection signal of the sensor.

[0047] The light source 54 is a part for irradiating light toward the photographing range of the camera 53. For example, an LED lamp is used for the light source 54. However, the light source 54 may be a fluorescent lamp or an incandescent lamp. The light source 54 is always lit when the camera 53 performs photographing.

[0048] The hood 55 is a box-shaped member that covers a part of the upper surface of the receiving conveyor 20. The hood 55 is formed of a light-shielding material. The hood 55 is disposed in the middle of the conveying path of the aggregate 9 by the receiving conveyor 20. The hood 55 has an inlet opening 551 and an outlet opening 552. The aggregate 9 conveyed by the receiving conveyor 20 is carried into the hood 55 through the inlet opening 551 and carried out of the hood 55 through the outlet opening 552.

[0049] The camera 53 and the light source 54 are disposed inside the hood 55. External ambient light is shielded by the hood 55. Thereby, the influence of the ambient light during the imaging by the camera 53 is suppressed. Also, inside the hood 55, by irradiating light from the light source 54, the aggregate 9 can be illuminated with a certain brightness. For this reason, even when the amount of solar radiation changes due to the season or the weather, the aggregate 9 can be imaged with a certain brightness. Also, it is possible to suppress the change in the shadow of the aggregate 9 due to the position of the sun.

[0050] The information processing unit 56 is a device that performs the learning process and the determination process described later. The information processing unit 56 is composed of a computer having a processor 561 such as a CPU or a GPU, a memory 562 such as a RAM, and a storage unit 563 such as a hard disk drive or an SSD. In the storage unit 563, a computer program for executing the learning process and the determination process described later is installed. Also, the information processing unit 56 is communicably connected to the operation panel 51, the display unit 52, the camera 53, the receiving conveyor 20, and the shuttle conveyor 30 described above.

[0051] FIG. 2 is a block diagram conceptually showing the functions realized in the information processing unit 56. As shown in FIG. 2, the information processing unit 56 has a learning unit 61, an estimation unit 62, a determination unit 63, and a control unit 64. The functions of these respective units are realized by the computer as the information processing unit 56 operating according to the computer program stored in the storage unit 563.

[0052] The learning unit 61 is a processing unit that generates a learning model M for estimating the aggregate type based on an image of the aggregate 9. The learning unit 61 performs machine learning using, as teacher data, an image of the sample aggregate 9 prepared for learning (hereinafter referred to as the "learning image") and the known aggregate type of the sample aggregate 9. As the machine learning algorithm, a convolutional neural network, which is a type of deep learning, is suitable. In the experiments conducted by the present inventors, when a convolutional neural network having a pooling layer that performs blurring processing on an image was adopted for the learning unit 61, a learning model M having sufficient estimation accuracy could be generated even for an image having blur in the conveyance direction.

[0053] However, the machine learning algorithm used by the learning unit 61 is not limited to a convolutional neural network, and may be other supervised machine learning algorithms such as a neural network, linear regression, decision tree, and support vector regression.

[0054] The learning unit 61 learns the relationship between the images of a large number of sample aggregates 9 and the aggregate types of the sample aggregates 9. As a result, a learning model M capable of outputting an estimation result of the aggregate type corresponding to the image of the aggregate 9 is generated. The learning unit 61 provides the generated learning model M to the estimation unit 62. The details of the learning process will be described later.

[0055] The estimation unit 62 is a processing unit that estimates the aggregate type from the image of the aggregate 9 using the above learning model M. The estimation unit 62 cuts out an image from the moving image D captured by the camera 53 and inputs it to the learning model M. Then, the learning model M outputs an estimation result E of the aggregate type based on the input image of the aggregate 9. The estimation unit 62 sends the estimation result E output from the learning model M to the determination unit 63.

[0056] The determination unit 63 acquires the information I of the aggregate type input to the operation panel 51. Further, the determination unit 63 acquires the estimation result E of the aggregate type by the estimation unit 62. Then, the determination unit 63 determines whether the information I of the aggregate type input to the operation panel 51 matches the estimation result E of the aggregate type by the estimation unit 62. Also, the determination unit 63 outputs the determination result R to the display unit 52. Thereby, on the display unit 52, the determination result R as to whether the aggregate type of the aggregate 9 conveyed by the receiving conveyor 20 matches the aggregate type specified by the operation panel 51 is displayed.

[0057] The control unit 64 controls the operation of the receiving conveyor 20 according to the determination result R of the determination unit 63. When the determination unit 63 determines that the information I of the aggregate type input to the operation panel 51 matches the estimation result E of the estimation unit 62, the control unit 64 continues the operation of the receiving conveyor 20. When the determination unit 63 determines that the information I of the aggregate type input to the operation panel 51 does not match the estimation result E of the estimation unit 62, the control unit 64 stops the operation of the receiving conveyor 20.

[0058] <1-3. About the learning process> Subsequently, the learning process executed in the above-described aggregate determination device 50 will be described. FIG. 3 is a flowchart showing the flow of the learning process. This learning process is executed in advance before the determination process of the aggregate 9 described later.

[0059] When performing the learning process, first, a sample aggregate 9 for learning is prepared (step S11). For example, when it is desired to learn four aggregate types A to D, sample aggregates 9 of the four aggregate types are prepared.

[0060] The operator feeds the prepared sample aggregates 9 one by one into the receiving conveyor 20 and operates the receiving conveyor 20. Then, while irradiating light from the light source 54, the camera 53 takes a moving image D of the sample aggregate 9 (step S12). When the shooting of the sample aggregate 9 of one aggregate type is completed, the operator feeds the sample aggregate 9 of the next aggregate type into the receiving conveyor 20. Then, the receiving conveyor 20 is operated, and the moving image D of the sample aggregate 9 is taken in the same manner as above. In this way, the moving images D of the sample aggregates 9 of all the prepared aggregate types are sequentially taken.

[0061] The moving image D taken by the camera 53 is composed of a large number of frame images. For example, when shooting for 5 minutes with a camera 53 having a frame rate of 30 frames / second, the moving image D obtained by the shooting is composed of 9000 frame images. In step S12, for all the prepared aggregate types, such a moving image D (an aggregate of frame images) is taken. Then, the moving image D obtained by the shooting is transmitted from the camera 53 to the information processing unit 56.

[0062] Inside the information processing unit 56, a large number of frame images included in the moving image D are stored in the storage unit 563 as learning images (step S13). As described above, the moving image D includes a large number of frame images. Therefore, the learning unit 61 can prepare a large number of learning images. FIG. 4 is a diagram showing an example of a frame image included in the moving image D. As shown in FIG. 4, due to the relationship between the frame rate of the camera 53 and the moving speed of the belt 23, the frame image is an image having blurring in the conveying direction.

[0063] The learning unit 61 classifies a large number of learning images into teacher images and validation images (step S14). For example, when 36000 learning images are obtained by shooting 4 types of sample aggregates 9 for 5 minutes each, 27000 of the 36000 learning images, which is 3 / 4 of them, are used as teacher images, and the remaining 9000 learning images, which is 1 / 4 of them, are used as validation images. However, the ratio of the teacher images to the validation images does not necessarily have to be 3:1.

[0064] The learning unit 61 uses a set of a teacher image and known aggregate types of the sample aggregate 9 shown in the teacher image as teacher data. Also, the learning unit 61 uses a set of a validation image and known aggregate types of the sample aggregate 9 shown in the validation image as validation data.

[0065] The learning unit 61 performs image processing such as blurring, horizontal flipping, and vertical flipping on the teacher image (step S15). By changing the teacher image through image processing, the range of images for learning is expanded. Thereby, the accuracy of machine learning can be improved. However, it is desirable that the image processing does not include image rotation processing. As described above, the teacher image is an image having blur in the conveyance direction. If rotation processing is performed, the direction of this blur changes, resulting in a decrease in the accuracy of machine learning. Note that among a large number of teacher images, there may be images on which no image processing is performed.

[0066] The learning unit 61 performs machine learning using a large number of teacher data after image processing (step S16). That is, the machine learning algorithm inputs a large number of teacher images into the learning model M one by one. Then, the parameters of the learning model M are adjusted so that the estimated result of the aggregate type output from the learning model M approaches the aggregate type of the input teacher image. The machine learning algorithm repeats such parameter adjustment processing for the number of prepared teacher data.

[0067] Thereafter, the machine learning algorithm uses a plurality of validation images to confirm the estimation accuracy of the aggregate type by the learning model M (step S17). Specifically, a validation image is input to the learning model M, and it is confirmed whether or not the estimated result of the aggregate type output from the learning model M matches the aggregate type of the input validation image. When the estimation accuracy of the aggregate type reaches a predetermined level (for example, a correct answer rate of 95% or more), the machine learning algorithm ends the learning process. Thereby, one learning model M capable of outputting an estimated result of the aggregate type based on the image of the aggregate 9 is generated.

[0068] In addition, when the estimation accuracy of the aggregate type does not reach the desired level, the processes of steps S12 to S17 described above may be repeated again. Further, after generating the learning model M, an image of a sample aggregate 9 different from the above-described teacher image and validation image is input to the learning model M, and it is further confirmed whether or not the estimated result of the aggregate type output from the learning model M matches the aggregate type of the sample aggregate 9 shown in the image.

[0069] Even if the aggregate 9 is of the same aggregate type, the surface color varies depending on the dry state, wet state, difference in the crushing location, etc. Therefore, in step S12 described above, for each aggregate type, the video D of the sample aggregate 9 in the dry state, the video D of the sample aggregate 9 in the wet state, and the video D of the sample aggregate 9 with different crushing locations may be obtained by shooting multiple times.

[0070] Also, among the numerous learning images input to the learning model M, images of the sample aggregate 9 in the dry state, images of the sample aggregate 9 in the wet state, and images of the sample aggregate 9 with different crushing locations may be included. For example, when using 9000 learning images for one aggregate type, a plurality of images of the sample aggregate 9 in the dry state, wet state, and with different crushing locations may be mixed among the 9000 images. Thereby, the learning unit 61 can learn various states such as the dry state, wet state, and difference in crushing location for each aggregate type. Therefore, the estimation accuracy of the aggregate type output from the learning model M can be further improved.

[0071] <1-4. Regarding the determination process of aggregates> Subsequently, the process of determining the suitability of the aggregate 9 input to the receiving conveyor 20 using the above learning model M when actually receiving the aggregate 9 will be described. FIG. 5 is a flowchart showing the flow of the determination process.

[0072] When receiving the aggregate 9, first, the aggregate 9 loaded on the truck 90 is put into the receiving hopper 10 (step S21). Subsequently, the operator inputs the information I of the aggregate type of the input aggregate 9 into the operation panel 51 (step S22). For example, by pressing the key 511 of the operation panel 51, the information I of the aggregate type is input. After that, the operator presses the key 512 for instructing the start of conveyance on the operation panel 51. Thereby, the operation of the receiving conveyor 20 is started (step S23).

[0073] The aggregate 9 supplied from the receiving hopper 10 to the upper surface of the belt 23 is conveyed obliquely upward toward the end pulley 22 as the belt 23 rotates. And after the operation of the receiving conveyor 20 is started, when a predetermined time has elapsed and the aggregate 9 passes below the camera 53, the camera 53 captures a moving image D of the aggregate 9 (step S24). Then, the camera 53 transmits the obtained moving image D to the information processing unit 56.

[0074] Inside the information processing unit 56, the estimation unit 62 cuts out one image (frame image) from the above-mentioned moving image D (step S25). Then, the estimation unit 62 inputs the cut-out image into the learning model M. Then, the learning model M outputs an estimation result E of the aggregate type based on the input image of the aggregate 9 (step S26). The estimation unit 62 sends the estimation result E output from the learning model M to the determination unit 63.

[0075] The determination unit 63 acquires the information I of the aggregate type input to the operation panel 51 in step S22 above. Also, the determination unit 63 acquires the estimation result E output from the learning model M in step S26 above. Then, the determination unit 63 determines whether or not the information I of the aggregate type input to the operation panel 51 and the estimation result E of the aggregate type by the estimation unit 62 match (step S27).

[0076] The determination unit 63 outputs the determination result R to the display unit 52. As a result, the determination result R indicating whether or not the aggregate type of the aggregate 9 conveyed by the receiving conveyor 20 matches the aggregate type specified by the operation panel 51 is displayed on the display unit 52 (step S28). The operator can recognize the determination result R by checking the display unit 52.

[0077] The control unit 64 checks whether the determination result R of the determination unit 63 in step S27 above is either "match" or "mismatch" (step S29). Then, when the determination result R indicates that the aggregate type information I input to the operation panel 51 matches the estimation result E of the estimation unit 62 (step S29: Yes), the control unit 64 continues the operation of the receiving conveyor 20. On the other hand, when the determination result R indicates that the aggregate type information I input to the operation panel 51 does not match the estimation result E of the estimation unit 62 (step S29: No), the control unit 64 stops the operation of the receiving conveyor 20 (step S30).

[0078] When the determination result R is "mismatch", there is a possibility that the aggregate 9 of an inappropriate aggregate type has been put into the receiving hopper 10, or that the aggregate type information I has been misinput to the operation panel 51. In either case, the aggregate 9 on the receiving hopper 10 may be conveyed to the wrong silo 40. In step S30 above, in such a case, the operation of the receiving conveyor 20 is stopped urgently. This stops the conveyance of the aggregate 9 and prevents the aggregate 9 from being conveyed to the wrong silo 40.

[0079] As described above, this aggregate determination device 50 photographs the aggregate 9 conveyed by the receiving conveyor 20, and uses the obtained image and the learning model M generated in advance by machine learning to estimate the aggregate type of the aggregate 9 during conveyance. Then, the aggregate determination device 50 determines whether or not the estimation result E of the aggregate type matches the aggregate type information I input to the operation panel 51. Thereby, it is possible to detect that the aggregate 9 may be conveyed to the wrong silo 40.

[0080] In particular, in the aggregate determination device 50 of the present embodiment, the aggregate 9 is photographed without stopping the receiving conveyor 20. Therefore, the image input to the learning model M is not a high-definition image but an image with blur. The camera 53 for photographing the aggregate 9 can use an inexpensive camera that is easy to obtain, rather than a high-speed camera with a high frame rate capable of acquiring a frame image without blur in the conveyance direction or a high-performance camera with a large number of pixels. Thereby, the cost of the camera 53 can be reduced. Further, by using an inexpensive camera 53 that is easy to obtain, the replacement cost when the camera 53 fails can also be suppressed. Therefore, an aggregate determination device 50 with excellent maintainability can be realized.

[0081] Also, in the aggregate discrimination device 50 of the present embodiment, a convolutional neural network that performs image blurring processing is adopted for generating the learning model M in the learning unit 61. Therefore, the image input to the learning model M does not necessarily have to be a high-definition image. That is, even an image with blur in the conveyance direction can obtain a learning model M with high estimation accuracy. Therefore, a learning model M with high estimation accuracy can be obtained while using an inexpensive camera 53 instead of a high-performance camera. The camera 53 may be, for example, a web camera with a frame rate of about 30 frames / second. Further, by suppressing the frame rate and the number of pixels, the storage capacity of the aggregate determination device 50 can be reduced. That is, an aggregate determination device with sufficient determination accuracy can be realized while having an inexpensive device configuration.

[0082] Incidentally, when the frame rate of the camera 53 is 30 frames per second, if the moving speed of the belt 23 of the receiving conveyor 20 is 50 m / min (the moving distance per frame is 2.78 cm / frame) or more, a frame image having blur in the conveying direction can be acquired. Further, when the moving speed of the belt 23 exceeds 120 m / min (the moving distance per frame is 6.67 cm / frame), the frame image obtained by shooting becomes so blurred that the outer shape of the aggregate cannot be discriminated. For this reason, it is advisable to adjust the relationship between the frame rate of the camera 53 and the moving speed of the belt 23 so that the moving distance per frame is in the range of 2.78 cm / frame or more and 6.67 cm / frame or less.

[0083] Further, in the aggregate determination device 50 of the present embodiment, within the hood 55, while irradiating light from the light source 54, shooting is performed by the camera 53. Thereby, the variation in the brightness of the image input to the learning model M can be suppressed. Therefore, the aggregate type can be estimated with higher accuracy.

[0084] <2. Second Embodiment (Moisture Sensor)> Subsequently, a second embodiment of the present invention will be described. FIG. 6 is a diagram showing the configuration of the aggregate conveying device 1 according to the second embodiment.

[0085] This embodiment is different from the above-described first embodiment in that the aggregate determination device 50 includes a moisture sensor 57. The moisture sensor 57 is a sensor that measures the moisture content on the surface of the aggregate 9 conveyed by the receiving conveyor 20. The moisture sensor 57 is disposed at a position above the receiving conveyor 20 and upstream of the camera 53. Further, the moisture sensor 57 is disposed within the hood 55 together with the camera 53 and the light source 54.

[0086] For the moisture sensor 57, for example, a near-infrared moisture meter is used. The near-infrared moisture meter measures the moisture content of the aggregate 9 by irradiating near-infrared rays toward the aggregate 9 and measuring the absorbance on the surface of the aggregate 9 based on the amount of reflected light. However, the moisture sensor 57 may be a moisture meter of another type. The moisture sensor 57 is communicably connected to the information processing unit 56. The measurement result of the moisture sensor 57 is transmitted to the information processing unit 56.

[0087] FIG. 7 is a flowchart showing the flow of the learning process in the second embodiment. As shown in FIG. 7, in this second embodiment, after preparing the sample aggregate 9, first, for the sample aggregate 9 in a dry state, the same processing as steps S12 to S17 described above is performed (steps S12A to S17A). Thereby, a first learning model M1 capable of outputting an estimation result E of the aggregate type based on the image of the aggregate 9 in the dry state is generated. Next, for the sample aggregate 9 in a wet state, the same processing as steps S12 to S17 described above is performed (steps S12B to S17B). Thereby, a second learning model M2 capable of outputting an estimation result E of the aggregate type based on the image of the aggregate 9 in the wet state is generated.

[0088] Note that the "dry state" refers to a state where the moisture content on the surface of the aggregate 9 measured by the above moisture sensor 57 is less than a predetermined value. Also, the "wet state" refers to a state where the moisture content on the surface of the aggregate 9 measured by the above moisture sensor 57 is equal to or more than a predetermined value.

[0089] FIG. 8 is a flowchart showing the flow of the determination process in the second embodiment. As shown in FIG. 8, in this second embodiment, the same processing as steps S21 to S25 is performed as in the first embodiment to obtain an image of the aggregate 9. However, immediately before the aggregate 9 is photographed by the camera 53, the moisture content on the surface of the aggregate 9 is measured by the moisture sensor 57 (step S24A).

[0090] Thereafter, the estimation unit 62 determines whether the aggregate 9 conveyed by the receiving conveyor 20 is in a dry state or a wet state (step S31). When the moisture content measured by the moisture sensor 57 is less than the predetermined value described above, the estimation unit 62 determines that the aggregate 9 is in a dry state (step S31: Yes). When the moisture content measured by the moisture sensor 57 is greater than or equal to the predetermined value described above, the estimation unit 62 determines that the aggregate 9 is in a wet state (step S31: No).

[0091] When the aggregate 9 is in a dry state, the estimation unit 62 selects the first learning model M1 (step S32). Also, when the aggregate 9 is in a wet state, the estimation unit 62 selects the second learning model M2 (step S33).

[0092] When the first learning model M1 is selected, the estimation unit 62 inputs the image of the aggregate 9 into the first learning model M1. Then, based on the input image, the first learning model M1 outputs an estimation result E of the aggregate type (step S26). Also, when the second learning model M2 is selected, the estimation unit 62 inputs the image of the aggregate 9 into the second learning model M2. Then, based on the input image, the second learning model M2 outputs an estimation result E of the aggregate type (step S26).

[0093] Thereafter, the aggregate determination device 50 performs the processes of steps S27 to S30 in the same manner as in the first embodiment.

[0094] As described above, in this second embodiment, the aggregate determination device 50 includes a moisture sensor 57. When the aggregate 9 is in a dry state, the first learning model M1 with parameters adjusted for the dry state is used. Also, when the aggregate 9 is in a wet state, the second learning model M2 with parameters adjusted for the wet state is used. By using different learning models M1 and M2 for the dry state and the wet state in this way, the aggregate type can be estimated more accurately.

[0095] <3. Third Embodiment (Majority Voting)> Next, a third embodiment of the present invention will be described. FIG. 9 is a flowchart showing the flow of the determination process in the third embodiment. As shown in FIG. 9, in this third embodiment, similar to the first embodiment, the processes of steps S21 to S24 are performed to obtain the moving image D of the aggregate 9.

[0096] The estimation unit 62 cuts out odd-numbered (odd numbers of 3 or more) images from the above moving image D (step S25). Then, the estimation unit 62 inputs the cut-out odd-numbered images into the learning model M respectively. Then, the learning model M outputs an odd number of estimation results E corresponding to each of the input odd-numbered images of the aggregate 9 (step S26). The estimation unit 62 sends the odd number of estimation results E output from the learning model M to the determination unit 63.

[0097] Subsequently, the determination unit 63 compares whether each of the above odd-numbered estimation results E matches the information I of the aggregate type input to the operation panel 51. Then, the determination unit 63 makes a final determination by a majority vote of the comparison results (step S27). That is, if the number of "matches" is more than the number of "mismatches" among the odd-numbered comparison results, the determination unit 63 sets the final determination result R to "match". Also, if the number of "mismatches" is more than the number of "matches" among the odd-numbered comparison results, the determination unit 63 sets the final determination result R to "mismatch".

[0098] After that, the aggregate determination device 50 performs the processes of steps S28 to S30 in the same manner as in the first embodiment.

[0099] As described above, in this third embodiment, an odd number of estimation results E are output from the learning model M based on the odd-numbered images of the aggregate 9. Then, based on the odd number of estimation results E, it is determined by a majority vote whether the aggregate 9 input to the receiving conveyor 20 matches the aggregate type input to the operation panel 51. Thereby, a more accurate determination result R can be obtained.

[0100] Note that the aggregate determination device 50 may have three or more odd-numbered cameras 53. Then, one image may be cut out from each of the videos D captured by the odd-numbered cameras 53 to obtain an odd number of images.

[0101] <4. Fourth Embodiment (Additional Learning)> Subsequently, a fourth embodiment of the present invention will be described. In this fourth embodiment, while performing the above-described determination process using the once-generated learning model M, additional machine learning is appropriately performed, which is different from the first embodiment above. FIG. 10 is a flowchart showing the flow of processing when performing additional machine learning.

[0102] In the example of FIG. 10, after the determination result R is displayed on the display unit 52, the operator determines whether the displayed determination result R is correct. Then, the correct / incorrect information indicating whether the determination result R is correct is input to the operation panel 51 (step S34).

[0103] For example, when the determination result R of "discrepancy" is displayed on the display unit 52, the operator checks whether the aggregate type of the aggregate 9 input to the receiving conveyor 20 and the aggregate type information I input to the operation panel 51 are indeed discrepant. And when they are indeed discrepant, the operator inputs to the operation panel 51 that the determination result R by the determination unit 63 is correct. Also, when the aggregate type of the aggregate 9 input to the receiving conveyor 20 and the aggregate type information I input to the operation panel 51 match, the operator inputs to the operation panel 51 that the determination result R of the determination unit 63 is incorrect.

[0104] The above-described correct / incorrect information input to the operation panel 51 is transmitted to the information processing unit 56. The information processing unit 56 accumulates, as teacher data for additional learning, a set of the image cut out in step S25, the estimation result in step S26, and the correct / incorrect information input in step S34 in the storage unit 563 (step S35). The information processing unit 56 accumulates such teacher data each time the receiving conveyor 20 accepts the aggregate 9.

[0105] Note that in step S35, not only the image (frame image) cut out in step S25 but also a moving image D of a predetermined time (for example, 5 minutes) including the image may be included in the teacher data for additional learning. By doing so, a large number of frame images included in the moving image D can be used as images for additional learning. Therefore, a large amount of teacher data can be accumulated in a short time.

[0106] After that, the learning unit 61 of the information processing unit 56 performs additional machine learning using the accumulated teacher data at a predetermined timing (step S36). The timing for performing the additional machine learning may be, for example, when the ratio of the above-described correct / error information being "error" becomes higher than a predetermined threshold. The learning unit 61 updates or re-creates the learning model M using only the teacher data for additional learning or the initial teacher data with the teacher data for additional learning added thereto. Thereby, the estimation accuracy of the aggregate type by the learning model M can be maintained at a predetermined level.

[0107] Also, when measuring the moisture content on the surface of the aggregate 9 with the moisture sensor 57 as in the second embodiment described above, the measured moisture content may be included in the teacher data for additional learning.

[0108] <5. Modification Example> As described above, the first to fourth embodiments of the present invention have been described, but the present invention is not limited to the above-described embodiments.

[0109] In the above embodiment, the learning unit 61 has performed image processing such as blurring, horizontal flipping, and vertical flipping on the frame image. However, the frame image may be used as it is as a learning image without performing image processing.

[0110] In the above-described embodiment, the camera 53 captured the moving image D of the aggregate 9 conveyed by the receiving conveyor 20. However, the camera 53 may capture a still image of the aggregate 9 conveyed by the receiving conveyor 20. Even when capturing a still image instead of the moving image D, an image including blur can be obtained by capturing the image without stopping the conveyance of the aggregate 9. That is, the camera 53 may obtain, as a learning image, an image including blur by capturing a still image of the conveyed aggregate 9, not limited to the moving image D.

[0111] In the above-described embodiment, when the determination result R by the determination unit 63 was "discrepancy", the receiving conveyor 20 was stopped. However, for example, a cut gate may be provided at the lower part of the receiving hopper 10, and when the determination result R is "discrepancy", the cut gate may be closed. Also, a surge bin capable of temporarily storing the aggregate 9 may be provided in the conveyance path, and when the determination result R is "discrepancy", the aggregate 9 may be conveyed to the surge bin. Further, a gate may be provided somewhere in the conveyance path to the silo 40, and when the determination result R is "discrepancy", the gate may be closed. That is, when the determination result R is "discrepancy", the control unit 64 may restrict the conveyance of the aggregate 9 to the silo 40 by some method.

[0112] In the above-described embodiment, the receiving conveyor 20 was inclined. However, the receiving conveyor 20 may not be inclined. That is, the receiving conveyor 20 may convey the aggregate 9 in the horizontal direction.

[0113] In the above-described embodiment, the computer as the information processing unit 56 was arranged inside the concrete plant. However, part or all of the functions of the information processing unit 56 may be arranged outside the concrete plant. For example, the computer as the information processing unit 56 may be arranged at a remote location and be able to communicate with the operation panel 51, the display unit 52, and the camera 53 via the Internet.

[0114] In the above-described embodiment, the determination result R of the determination unit 63 was displayed on the display unit 52. However, when the determination result R of "mismatch" is output from the determination unit 63, a warning lamp may be lit or a warning buzzer may be sounded. That is, the "result output unit" in the present invention may be a warning lamp, a warning buzzer, or the like.

[0115] In the above-described embodiment, an example of determining the aggregate type in the receiving conveyor 20 of the concrete plant was described. However, the present invention can also be used in a quarry that ships and supplies aggregate 9 to the concrete plant. That is, the aggregate determination device of the present invention may determine the aggregate type in a conveyor installed at a quarry.

[0116] Also, the respective elements appearing in the above-described embodiment and modification examples may be appropriately combined within a range where no contradiction occurs.

Industrial Applicability

[0117] The present invention can be used to determine the aggregate type in a concrete plant or a quarry.

Explanation of Signs

[0118] 1 Aggregate conveying device 9 Aggregate 10 Receiving hopper 20 Receiving conveyor 30 Shuttle conveyor 40 Silo 50 Aggregate determination device 51 Operation panel 52 Display unit 53 Camera 54 Light source 55 Hood 56 Information processing unit 57 Moisture sensor 61 Learning unit 62 Estimation unit 63 Determination unit 64 Control unit 90 Truck D Video E Estimation result I Aggregate type information M Learning model M1 First learning model M2 Second learning model R Judgment result

Claims

1. An aggregate determination device for determining the suitability of aggregates fed onto a conveyor, comprising: a camera for photographing the aggregates conveyed by the conveyor; an operation panel into which information on the type of aggregate is input; an information processing unit communicably connected to the camera and the operation panel; wherein the information processing unit includes an estimation unit that inputs an image of the aggregate obtained by photographing with the camera into a learning model generated by machine learning and outputs an estimation result of the aggregate type from the learning model; a determination unit that determines whether or not the aggregate type input to the operation panel matches the estimation result; and the camera photographs a moving image of the aggregates conveyed by the conveyor, the moving image including a plurality of frame images having blurring in the conveying direction of the aggregates by adjusting the moving speed of the conveyor such that the moving distance per frame is 2.78 cm or more and 6.67 cm or less, the learning model being generated by performing machine learning using, as teacher data, the plurality of frame images having such blurring included in a moving image of sample aggregates prepared for learning, the aggregate determination device.

2. The aggregate determination device according to claim 1, wherein the information processing unit further includes a learning unit that generates the learning model by performing the machine learning. The aggregate determination device.

3. The aggregate determination device according to claim 2, wherein the learning unit performs the machine learning after performing image processing other than rotation on the frame images. The aggregate determination device.

4. The aggregate determination device according to claim 2 or claim 3, wherein the operation panel can input correct / incorrect information indicating whether or not the determination result by the determination unit is correct, and the learning unit updates the learning model by performing additional machine learning using, as teacher data, the image of the aggregate obtained from the camera, the estimation result, and the correct / incorrect information. The aggregate determination device.

5. The aggregate determination device according to any one of claims 1 to 4, further comprising a light source that irradiates light toward the imaging range of the camera. The aggregate determination device.

6. The aggregate determination device according to claim 5, further comprising a hood that covers a part of the upper surface of the conveyor, wherein the camera and the light source are disposed inside the hood. The aggregate determination device.

7. The aggregate determination device according to any one of claims 1 to 6, further comprising a result output unit that outputs the determination result of the determination unit. An aggregate determination device further having

8. An aggregate determination device for determining the suitability of aggregates fed into a conveyor, a camera that photographs the aggregates conveyed by the conveyor, an operation panel into which information on the aggregate type is input, an information processing unit communicably connected to the camera and the operation panel, comprising: The information processing unit an estimation unit that inputs an image of the aggregate obtained by photographing with the camera into a learning model generated by machine learning and outputs an estimation result of the aggregate type from the learning model; a determination unit that determines whether the aggregate type input to the operation panel matches the estimation result; having a moisture sensor that measures the moisture content on the surface of the aggregate conveyed by the conveyor further comprising: The information processing unit a first learning model generated by performing the machine learning on aggregates with a surface moisture content less than a predetermined value, a second learning model generated by performing the machine learning on aggregates with a surface moisture content greater than or equal to the predetermined value, having The estimation unit when the moisture content measured by the moisture sensor is less than the predetermined value, inputs an image of the aggregate photographed by the camera into the first learning model and outputs an estimation result of the aggregate type from the first learning model; when the moisture content measured by the moisture sensor is greater than or equal to the predetermined value, inputs an image of the aggregate obtained from the camera into the second learning model and outputs an estimation result of the aggregate type from the second learning model, an aggregate determination device.

9. The aggregate determination device according to any one of Claims 1 to 8, wherein the estimation unit inputs an odd number of images obtained from the camera into the learning model and outputs an odd number of estimation results corresponding to each of the odd number of images from the learning model, and the determination unit compares the aggregate type input to the operation panel with the odd number of estimation results and makes a determination by a majority vote of the comparison results, an aggregate determination device.

10. The aggregate determination device according to any one of Claims 1 to 9, wherein the information processing unit further has a control unit that restricts the conveyance of the aggregate to the silo when the determination unit determines that the aggregate type input to the operation panel does not match the estimation result an aggregate determination device.

11. An aggregate determination method for determining the suitability of aggregates fed into a conveyor, a) a step of inputting information on the aggregate type of the aggregates fed into the conveyor into an operation panel; b) a step of photographing the aggregate conveyed by the conveyor; c) a step of inputting an image of the aggregate obtained by the photographing in step b) into a learning model generated by machine learning and outputting an estimation result of the aggregate type from the learning model; d) a step of determining whether or not the aggregate type input in step a) matches the estimation result; which comprises: In step b), a moving image of the aggregate conveyed by the conveyor is photographed; The moving image includes a plurality of frame images having blurring in the conveying direction of the aggregate by adjusting the moving speed of the conveyor so that the moving distance per frame is 2.78 cm or more and 6.67 cm or less; The learning model is generated by performing machine learning using, as teacher data, a plurality of frame images having the blurring included in a moving image of sample aggregates prepared for learning. An aggregate determination method.

Images